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The  United  States  today  is  in  a  position  of  unprecedented  militaiy  strengtii 
and  national  security.  Its  strategic  nuclear  forcc.s.  together  with  its 
conventional  forces  numbering  over  400  battle  force  ships,  5,000  combat 
air  craft,  10,000  tanks,  and  numerous  other  combatant  platforms  equipped 
wi'ir  missiles,  bombs,  projectiles,  and  other  munitions,  are  a  visible  sign  of 
the  enormous  miliLaiy  power  it  possesses.  While  these  weapons  provide  a 
military  capability  that  no  other  country  is  in  a  position  to  successfully 
challenge,  their  acquisition  was,  in  many  cases,  fraught  with  significant 
problems.  Although  improvements  have  been  made  in  the  acquisition 
process  over  the  years,  the  same  kinds  of  problems  can  be  seen  in  today’s 
acquisitions. 

Why,  with  an  increased  emphasLs  on  sound  development  and  testing  of 
weapons,  do  we  still  witness  m^or  commitments  to  programs,  such  as  the 
B-2  bomber  and  the  Airborne  Self-Protection  Jammer,  without  first 
demonstrating  the  system  will  meet  critical  performance  requirements? 
Why,  with  improved  cost-estimating  policies  and  procedures,  do  we  still 
see  the  unit  costs  of  weapon  systems,  such  as  the  DDG-51  destroyer  and 
the  C-17  transport,  doubling?  Why,  with  the  increased  emphasis  on 
developing  systems  that  can  be  efficiently  produced  and  supported,  do  we 
have  weapons,  such  as  the  Advanced  Cruise  Missile  and  Apache 
helicopter,  that  encounter  costly  production  and  support  problems? 

Clearly,  problems  are  to  be  expected  in  mqjor  weapon  acquisitions,  given 
the  technical  risks  and  complexities  involved.  On  the  other  hand,  the 
historical  level  of  such  problems  should  not  be  accepted  as  the  norm.  As  a 
nation,  we  can  do  better.  The  question  is  how — what  can  be  done  to  foster 
better  acquisition  outcomes  with  more  success  than  we  have  experienced 
in  the  past?  Today,  the  prospects  for  constructive  change  arc  quite 
encouraging.  The  demise  of  the  Soviet  threat  euid  declines  in  defense 
budgets  have  created  a  unique  opportunity  to  effect  lasting  changes  in 
weapon  acquisitions.  Both  the  Department  of  Defense  and  the  Congress 
have  acted  upon  this  opportunity  and  have  shown  a  willingness  to  support 
the  type  of  changes  needed  to  improve  acquisition  outcomes.  The  current 
top  level  management  within  the  Office  of  the  Secretary  of  Defense  has 
taken  steps  in  an  attempt  to  discipline  the  acquisition  process  and  to 
redefine  the  basic  strategy  for  acquiring  weapons.  Key  Members  of 
Congress  have  pressed  for  reevaluations  of  weaporvs  programs  and  service 
roles  and  functions. 

If  rhanvps  in  the  aerr.iisil.ion  of  weanrins  are  tn  he  of  p  la.eit.ino  nature,  we 
believe  acquisition  problems  also  need  to  be  looked  at  from  another 
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perspective — as  the  consequences  of  a  way  of  acquiring  weapons  that  has 
become  deeply  rooted  through  the  years.  We  refer  to  this  as  the  “culture," 
although  this  is  a  somewhat  imprecise  term.  Rather  than  defining  “culture” 
as  mindsets  and  attitudes,  we  use  the  term  to  describe  the  collective 
patterns  of  behavior  exhibited  by  tlie  numerous  participants  in  the 
acquisition  process  and  the  incentives  for  that  behavior.  These 
participants  include  the  vairious  components  of  the  Department  of 
Defense,  the  Congress,  industry,  and  critics.  By  looking  at  acquisition 
problems  from  the  perspective  of  culture,  one  can  gain  greater  insight  into 
why  such  problems  are  resilient — reasons  that  go  beyond  technical  risks, 
estimating  errors,  and  shortcomings  in  oversight. 

This  culture  has  evolved  as  the  acquisition  process  has  become  a  vehicle 
for  meeting  the  diverse  needs  of  participants  through  the  steady  initiation 
and  sustainment  of  weapon  programs.  These  needs,  as  translated  into 
weapon  systems,  transcend  the  filling  of  voids  in  military  capability  at 
minimal  cost;  they  involve  the  definition  of  roles  and  missions,  the 
justifications  of  budget  levels  and  shares,  service  reputations, 
organizational  influence,  the  industrial  base,  jobs,  and  careers. 

Individually,  participants  act  rationally,  for  they  see  their  needs  as  aligned 
with  the  national  interests.  However,  collectively,  these  needs  create  an 
environment  that  encourages  “selling”  programs,  along  with  undue 
optimism,  parochialism,  and  other  compromises  of  >od  judgment.  For 
example,  is  it  rea.sonable  to  expect  program  sponsors  to  present  objective 
risk  assessments,  report  recdistic  cost  estimates,  or  perform  thorough  te.sts 
of  prototypes  when  such  measures  expose  programs  to  disruption, 
deferral,  or  even  cancellation?  Because  it  is  not  reasonable  to  do  so,  a 
level  of  cost  growth,  performance  problems,  schedule  delays,  and 
difficulties  with  production  and  field  support  is  essentially  embedded  in 
acquisition  programs  from  the  begirming.  In  this  light,  blaming  problems 
on  weapons  program  managers  or  any  other  single  participamt  overlooks 
the  point  that  these  problems  are  the  collectwe  responsibility  of  all 
participants. 

This  report  reflects  on  the  m^or  acquisition  issues  addressed  in  our  work 
over  the  past  15  years.  Upon  taking  note  of  the  similarity  of  the  issues 
during  those  years,  the  report  explores  the  cultural  side  of  acquisition 
problems  to  suggest  ways  to  ameliorate  such  problems  in  the  future.  These 
suggestions  are  outlined  in  chapter  5.  This  report  is  not  presented  as  the 
singular  correct  view  of  the  i.ssne.s  di.scii.s.sed'.  nOr  does  it  attempt  to  cover 
all  acquisition  problems.  However,  it  does  contend  that  making 
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fundamental  improvements  in  acquisitions  Vv  ill  require  attacking  the 
cultural  dimension  of  the  “problem.” 

Changes  of  the  type  needed  will  not  come  easily.  They  must  be  directed  .at 
the  system  of  incentives  that  has  become  self-sustaining  and  very  difficult 
to  uproot.  The  incentives  that  motivate  the  participants  must  be  realigned 
with  better  program  outeomes.  If  we  expect  program  sponsors,  for 
example,  tc  be  forthright  about  program  alternatives,  costs,  and  risks, 
such  candor  must  be  rewarded,  aiad  parochialism  and  undue  optimism 
penalized.  Ultimately,  change  will  occur  only  through  the  collective  actio  n 
of  acquisition  participants,  particularly  within  the  Department  of  Defense 
and  the  Congress,  for  it  is  their  actions  that  dictate  the  incentives  that 
drive  the  process.  It  is  our  hope  that  this  report  will  help  to  illuminate  the 
cultural  changes  needed  to  meet  the  continuous  challenge  of  improving 
acquisition  outcomes. 


Charles  A.  Bowsher 
Comptroller  General 
of  the  United  States 
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iniroaucuon 


Weapon  system  acquisitions  involve  highly  leveraged  decisions  with 
significant  consequences  for  the  budget  and  for  national  security.  A  few 
discrete  decisions  on  weapon  systems  can  commit  the  nation  to  spending 
tens  of  billions  of  dollars,  can  represent  national  security  policy  choices, 
can  shape  how  wars  are  fought,  and  can  affect  the  relative  roles  and 
functions  of  the  military  services.  Weapon  systems  are  possibly  the  most 
visible  symbols  of  military  strength  £ind  the  reputation  of  the  Department 
of  Defense  (dod).  They  have  come  to  epitomize  both  the  successes  and  the 
failures  of  the  defense  establishment. 

This  study  focuses  on  acquisition  problems  that  have  persisted  over  time. 
The  sweeping  changes  in  national  security  stemming  from  the  dissolutioir 
of  the  Warsaw  Pact  and  the  Soviet  Union  have  fostered  a  climate 
conducive  to  confronting  acquisition  problems  in  a  fundamental  way.  We 
have  taken  a  retrospective  look  at  the  hundreds  of  weapon  system  reviews 
we  have  conducted  over  the  past  15  years  to  extract  lessons  learned  for 
future  acquisitions.  Our  focus  here  is  on  those  cultural  or  motivational 
factors  that  can  inhibit  the  application  of  sound  acquisition  practices. 
Taking  a  fundamental  approach  to  solving  acquisition  problems  is 
especially  critical  as  decisions  aie  made  on  a  series  of  next-generation 
weapons  in  the  coming  years.  Experience  has  shown  that  many  problems 
originate  early  in  programs  and  must  be  confronted  when  the  earliest 
decisions  are  made.  If  a  “business  as  usual”  approach  is  taken,  crucial 
compromises  on  the  defense  budget  are  likely  to  unravel  if  key  acquisition 
programs  cannot  be  executed  within  planned  funding  levels  and  time 
frames. 


Weapon  Acquisitions 
Will  Continue  to 
Figure  Prominently  in 
Budget  and  Policy 
Decisions 


Wxiile  Don  has  proposed  considerable  cutbacks  in  the  number  and  size  of 
its  weapons  acquisition  programs  over  the  next  several  years,  the  financial 
cind  policy  implications  of  the  programs  to  be  pursued  will  continue  to  be 
significant.  The  discretionary  nature  of  the  research  and  development  and 
procurement  accounts  makes  these  segments  of  the  defense  budget  the 
subject  of  intense  debate,  particularly  in  times  of  declining  resources.  The 
enonnous  funding  implications  of  weapons  decisions  alone  command 
national  attention.  Even  with  the  reduced  threat,  dou  has  proposed 
spending  about  $100  billioi.  a  year  on  the  development  and  procurement  of 
v/eapons  and  related  items  through  the  late  1990s.  It  is  no  coincidence, 
therefore,  that  tne  administration’s  1992  proposals  to  reduce  the  defense 
budget  relied  heavily  on  cuts  to  several  major  weapons  programs — cuts 
that  have  since  prompted  debate  and  changes  within  the  Congress. 
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quisition  Funds  Are 
iiong  the  Most 
scretionaiy  in  the  DOD 
idget 


Chapter  1 
Introduction 


At  the  highest  levels,  weapon  systems  represent  fiscal  policy  choices — in 
particular,  the  level  of  defense  spending.  Weapon  system  decisions  have 
taken  on  added  signif.cance  as  they  have  come  to  represent  key  choices 
on  how  to  implement  a  national  security  strategy  that  is  responsive  to  the 
changing  world  situation.  Weapon  systems  are  intert'vined  with  issues 
such  as  (1)  whether  the  United  States  should  implement  a  global  defense 
policy  with  the  Str  ategic  Defense  Initiative,  (2)  how  and  where  the  armed 
forces  should  be  prepared  to  fight,  (3)  whether  weapon  programs  should 
be  used  as  instruments  for  protecting  the  industrial  base  and  jobs, 

(4)  whether  dod  should  adopt  an  acquisition  strategy  of  carrying  new 
weapons  only  to  the  prototype  stage  rather  than  to  production,  and 

(5)  whether  modification  programs  should  be  emphasized  over  new 
weapons.  As  the  administration  and  the  Congress  consider  these  issues,  it 
is  clear  that  decisions  on  an  individual  weapon  system  mean  much  more 
than  simply  buying  a  piece  of  equipment. 


Weapon  systems  will  continue  to  be  debated  as  difficult  decisions  are 
made  on  how  to  accommodate  declining  defense  budgets.  In  the  dod 
budget,  the  Procurement  and  Research,  Development,  lest  and  Evaluation 
(rdt&e)  accounts — from  which  acquisitiorrs  are  funded — are  the  most 
discretionary;  when  there  are  changes  in  defense  spending,  these  accounts 
generally  reflect  those  changes  the  most.  Figure  1.1  illustrates  the  pattern 
of  these  changes. 
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Introductioii 


igure  1.1:  Annual  Percentage  Changes  in  Total  DOD  Budget  Authority  and  the  Sum  of  Procurement  and  RDT&E  Accounts 
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Change  In  Procurement  and  RDT&E 
“  ”  Change  In  total  DOD  budget  authority 


Notes:  Amounts  are  standardized  as  constant  fiscal  year  1992  dollars.  Data  for  fiscal  years  1992 
through  1997  are  estimates. 

Source:  GAO  analysis  of  DOD  budget  data, 


A  statistical  relationship  exists  between  changes  in  the  combined 
Procurement  2ind  RDTfiE  accounts  and  changes  in  the  total  dod  budget. 
Fluctuations  in  the  combined  Procurement  and  rdtae  accounts  are 
matched  by  similar,  less  drastic  changes  in  the  total  dod  budget.  Other 
large  eiccounts,  such  as  Personnel  and  Operations  and  Maintenance,  have 
not  been  as  volatile;  they  are  largely  determined  by  previous  decisions  on 
force  structure  and  size  and  on  qurrent  weapon  inventories. 


Page  8 


GA<VNSIAD-93>16  Lasting  Change  in  Weapons  Acquisition 


Chapter  1 
Introduction 


Decisions  on  Weapon 
System  Acquisitions  Affect 
a  Significant  Portion  of  the 
Total  DOD  Budget 


Procurement ;  nd  rdt&e  make  up  a  significant  portion  of  the  total  dod 
budget  and  will  likely  continue  to  do  so  in  the  future.  Between  fiscal  years 
1956  and  1991,  these  accounts  represented  an  average  of  37.6  percent  of 
the  total  DOD  budget.  However,  as  illustrated  in  figure  1.2,  that  portion  of 
the  total  DOD  budget  encompassed  by  Procurement  and  bdt&e  has  not  been 
a  constant;  it  has  fluctuated  over  time  from  about  30  to  nearly  45  peicent. 


Figure  1 .2:  Sum  of  Procurement  and  RDT&E  Accounts  as  a  Percentage  of  Total  DOD  Budget  Authority 
50  P«rc«nt 


Notes:  Amounts  are  standardized  as  constant  fiscal  year  1992  dollars.  Data  for  fiscal  years  1992 
through  1997  are  estimates. 

Source:  GAO  analysis  of  DOD  budget  data. 


There  have  been  two  major  liigh  points  in  budget  authority,  or  “bcw 
waves,”  occurring  during  the  eeirlier  stages  of  the  war  in  Vietnam  and  the 
defense  buildup  of  the  1980s.  During  these  two  waves,  the  United  States 
procured  previous  generations  of  major  weapon  systems.  Whether  another 
such  “bow  wave”  will  take  place  with  the  next  generation  of  v/eapon 
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Chapter  I 
Introduction 


systems  in  the  late  1990s  ai\d  early  2000s  depends  on  funding  decisions  to 
be  made  over  the  next  several  years. 


Decisions  on  Weapon 
Systems  Represent 
Choices  About  Policy, 
Strategy,  and  Roles 


The  acquisition  of  weapon  systems  or  types  of  systems  not  only  obligates 
large  amounts  of  funding,  but  represents  defense  policy  decisions. 
Weapons  research  and  acquisitions  Ccin  shape  milit^uy  strategy  and  can 
influence  the  roles  and  functions  of  the  individual  militaiy  services  and 
weapon  systems.  For  example, 

•  procuring  Pershing  II  and  ground-launched  cruise  missiles  demonstrated 
an  increased  commitment  to  the  forward  deployment  of 
intermediate-range  nuclear  weapons  in  Western  Europe; 

•  the  Strategic  Defense  Initiative’s  first  proposed  deployment,  to  counter  a 
massive  Soviet  attack  with  space-based  weaponry,  represented  a  new 
strategy  for  the  deterrence  of  nuclear  war;  and 

•  a  commitment  to  buy  heavy  tanks  can  also  be  a  commitment  to  forward 
deployment  and  prepositiorung,  whereas  the  purchase  of  light  vehicles 
suggests  a  conmiitment  to  rapid  deployment. 

In  addition,  dod’s  decision  to  develop  a  higher  yield  warhead  for  the 
Trident  submarine-launched  b^lllistic  missile  expanded  the  weapon’s  role. 
The  more  lethal  warhead,  together  with  significant  improvements  in 
missile  accuracy,  broadened  the  strategic  role  of  the  Navy  and  its  Trident 
submarine  force  from  one  suitable  only  for  nonhardened  military  and 
urban  industrial  targets  to  one  capable  of  destroying  hardened  targets 
such  as  missile  silos  and  underground  command  and  control  centers. 
Similarly,  the  role  of  the  manned  peneU'ating  strategic  bomber  was 
tlireatened  in  1977,  when  the  President  terminated  production  of  the  B-IA 
bomber  in  favor  of  accelerated  development  and  deployment  of  the 
Air-Launched  Cruice  Missile, 


GAO  Has  a 

Considerable  History 
of  Weapon  System 
Evaluations 


c.AO  has  been  involved  in  the  evaluation  of  weapon  systems  and  the 
acquisition  process  for  over  30  years.  During  the  late  1960s,  we  began  to 
place  more  emphasis  on  reviews  of  major  weapon  systems.  In 
February  1970,  our  first  report  on  the  status  of  57  major  dod  acquisition 
pregrams  then  in  development  or  procurement  was  issued.*  In  March  1971 
we  issued  our  first  report,  on  the  i)(.)i)  acquisition  process  itself,  responding 
to  Congress’  desire  for  complete  and  impartial  information  about  major 


^Status  (iF  the  Ac-quisitioti  of  Splpcted  MzQor  Weapon  Systoms  (BdG3058,  6,  1970). 
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weapon  systems  to  facilitate  critical  acquisition  decisions.  -  This  report, 
presented  the  rationale  for  performing  such  evaluations — a  rationale  that 
remains  pertinent  today: 

The  large  investment  required  in  recent  years  for  acquisition  of  lu^qor  weapons  has 
impacted  heavily  on  the  resources  available  for  other  national  goals  and  priorities. 

Acquiring  these  niEyor  weapons  involves  substantial  long-raiige  coinrnitinent  of  future 
expenditures.  Because  of  deep  concern  in  the  Congress  on  these  matters  and  because  of 
evidence  that  the  weapon  systems  acquisition  process  has  serious  weaknesses,  the  General 
Accounting  Office  (gao)  has  undertaken  to  provide  the  Congress  and  the  Department  of 
Defense  (dod)  with  a  continuing  series  of  appraisals  of  those  factors  most  closely  related  to 
effective  perfo nuance  in  procuring  mzyor  weapons. 

Since  that  time,  our  audits  have  resulted  in  over  900  repoits  and 
testimonies  on  virtually  all  aspects  of  weapon  system  acquisitions.  Tliis 
work  .has  covered  weapons  in  all  phases,  from  concept  fonnulation  to 
readiness  in  the  field,  including  muitiservice  programs  and  joint  ventures 
with  foreign  countries.  It  has  included  evaluations  of  the  acquisition 
process  itself  as  well  as  numerous  systemic  issues,  such  as  tlu’eat 
assessments,  cost  estimating,  competition,  test  and  evalualiuu,  pcisoimel 
practices,  logistics  management,  producibility,  acquisition  strategies, 
contracting,  budgeting,  and  programming.  It  was  because  of  this 
experience  that  in  November  1988,  we  advised  the  new  administration  and 
the  Congress  on  what  we  saw  as  the  critical  issues  facing  the  management 
of  defense  programs,  along  with  recommended  actions  to  correct 
problems.  ^  In  chis  study,  we  again  draw  upon  our  acquisition  experience  tc 
advise  congressional  and  executive  policymakers  as  they  make  tiie 
decisions  that  will  reshape  the  nation’s  defense  posture. 

Our  approach  to  evaluating  defense  acquisitions  generally  involves 
assessing  program  results  against  the  criteria  embodied  in  law,  non 
regulations,  and  individued  program  requirements  sucli  as  de.sign 
specifications.  In  essence,  we  hold  programs  to  Don’s  own  criteria,  except 
when  criteria  are  found  wanting.  The  basic  policies  and  procedures  of 
sound  acquisition  management  are  captured  in  ix)i)’s  acquisition 
regulations  and  in  the  Office  of  Management  and  Budget’s  Circular  A-109, 
“Mjyor  System  Acquisitions.”  We  tailor  our  reviews  of  individual  weapon 
systems  to  tlie  standards  applicable  to  the  particular  acquisition  phase  the 
program  i.s  in.  These  phases,  punctuated  by  key  decision  or  milestone 
points,  are  depicted  in  figure  1.3. 


'^Acquisition  of  M^jor  Wt^apon  SystAims  ( B- 1  (>3058 ,  M ar .  18,  1  [ )7 1 J . 
‘T^’ansition  Stines:  Dofense  Issues  (GAO/OC/’-89-9TH,  Nov.  ni88). 
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Figure  1.3:  DOD’s  Systems  Acquisition 
Process 


Noie:  A  tourin  miiosionu  rndy  uu  nt!utJbi.dry  lO  uotSmiinG  Vvr 
needed  for  the  weapon  in  production. 


imnrtarlQQ  pro 
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Since  our  reports  of  30  years  ago,  our  work  has  expanded  into  areas  such 
as  examining  policy  options  for  future  levels  of  defense  spending  eind 
evaluating  the  broad  in.plications  of  decisions  on  the  Strategic  Defense 
Initiative,  the  nuclear  triad,  and  the  naval  force  structure.  Developing  morf 
"hands-on”  information  on  programs,  we  have  responded  to  the  Congress’ 
desire  that  we  personally  witness  key  tests  and  report  our  independent 
views  directly  to  Committees  and  Members.  Throughout  these  evaluations 
we  have  favored  solving  problems  and  reducing  risks  before  proceeding 
with  programs.  In  addition,  our  work  reflects  our  view  that 
decisionmakers  must  be  armed  with  critical  and  reliable  information  at  the 
time  decisions  are  made.  Consequently,  we  attempt,  as  much  as  possible, 
to  r'ign  our  work  with  the  key  decisions  to  be  made,  particularly  regarding 
the  congressional  budget  and  oversight  t^ycle. 


)bjectives,  Scope, 
.nd  Methodology 


Although  key  reform  measures  have  led  to  improvements  in  weapon 
system  acquisition,  many  probleras  persist.  Recent  changes  in  the 
geopolitical  atmosphere  of  the  world,  combined  with  mounting  U.S. 
budgetary  concerns,  promde  the  opportunity  and  the  need  to  address 
persistent  acquisition  problems.  The  objective  of  this  study  is  therefore 
not  simply  to  catalog  the  findings  of  gao’s  previous  weapons  acquisition 
work,  but  rather  to  identify  the  underlying  factors  that  we  believe 
contribute  significantly  to  recurring  acquisition  difficulties. 

To  accomplish  this,  we  examined  our  past  work,  concentrating  on  the  last 
.15  years  of  reviews  of  weapon  systems  and  the  acquisition  process.  We 
performed  this  analysis  from  June  1991  through  September  1992.  In 
addition  to  reexamining  our  own  work,  v/e  drew  upon  the  analyses  of 
other  organizations,  including  various  reform  initiatives,  and  exchanged 
views  with  numerous  individuals  knowledgeable  in  acquisitiori 
management,  including  several  former  officials  from  the  Office  of  the 
Secretary  of  Defense.  Using  this  “corporate  knowledge”  to  extract  lessons 
learned  and  to  concentrate  on  the  cultural  reasons  behind  persistent 
problems  has  required  making  some  judgments. 

Although  we  did  not  obtain  official  agency  comments  on  this  report,  we 
did  discuss  its  contents  with  cognizant  officials  within  the  Office  of  the 
Secretary  of  Defense.  These  officieds  did  not  agree  that  acquisition 
problems  could  be  ascribed  to  a  culture.  Rather,  they  believed  that 
acquisition  problems  were  more  attributable  to  a  lack  of  discipline  and  to 
the  pressures  of  the  Cold  W2ir.  The  officials  noted  that  they  have  taken 
m^jor  steps  to  iiriprove  the  discipline  in  the  acquisition  process  and 
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consequently  believed  that  our  report  did  not  accurately  portray  the  status 
of  weapon  acquisitions  today. 

In  general,  we  support  the  actions  taken  by  the  Office  of  the  Secretary  of 
Defense  to  improve  acquisitions.  As  noted  elsewhere  in  the  report,  we  are 
encouraged  by  the  willingness  of  the  current  top-level  officials  to  forge 
significant  change.  We  also  believe  that  if  these  actions  are  sustained,  they 
will  lead  to  better  acquisition  outcomes.  However,  on  the  basis  of  our  past 
and  current  work,  we  believe  there  is  a  cultural  dimension  of  acquisition 
problems  that  helps  to  explain  their  persistence.  This  dimension  has  to  do 
with  the  fact  that  for  rational  reasons  the  interaction  of  the  acquisition 
participants  engenders  and  perpetuates  these  problems.  In  our  opinion, 
this  aspect  of  the  problem  goes  beyond  discipline  and  beyond  the  control 
of  DOD,  as  it  involves  other  participants,  such  as  the  Congress.  Yet  we  do 
not  believe  that  acknowledging  the  cultural  dimension  of  acquisition 
problems  is  incompatible  with  the  actions  taken  by  the  Office  of  the 
Secretary  of  Defense.  In  fact,  we  believe  it  can  complement  such  actions  iir 
fostering  better  acquisition  outcomes. 

In  the  next  three  chapters  we  discuss  (1)  the  persistent  problems  faced  by 
weapon  system  acquisitions,  (2)  the  sources  of  problems  inherent  in  the 
acquisition  culture,  and  (3)  the  resi.stance  of  the  culture  to  reform.  Then,  in 
chapter  5,  we  offer  some  suggestions  on  how  the  acquisition  culture  can 
be  improved  by  participants,  particularly  within  the  Department  of 
Defense  and  the  Congress. 

All  photographs  in  this  report  were  provided  by  dod,  with  the  exception  of 
the  photograph  of  the  Advanced  Cruise  Missile,  which  was  provided  by 
General  Dynatnic-s, 
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The  weapon  system  problems  we  have  reported  in  the  past  2  years  mirror 
those  we  reported  in  the  1970s  and  the  1980s.  This  does  not  imply  that  the 
history  of  weapon  acquisitioru;  amounts  to  a  string  of  bad  programs.  The 
point  is  that,  despite  conscious  attempts  to  improve  the  acquisition 
process,  weapons  still  cost  more,  take  longer  to  field,  often  encounter 
performance  problems,  and,  in  many  instances,  are  difficult  to  produce  or 
support.  The  persistence  of  these  problems  reflects  the  fact  that  the 
design,  development,  and  production  of  m^jor  weapon  systems  are 
extremely  complex  technical  processes  that  must  operate  within  equally 
complex  budget  and  political  processes.  If  not  well  conceived,  planned, 
managed,  funded,  and  supported,  problems  such  as  cost  growth,  schedule 
delays,  and  performance  shortfalls  can  easily  befall  a  program.  Even 
properly  run  programs  can  experience  problems  that  arise  from 
unknowns,  such  as  technical  obstacles  and  changes  in  the  threat.  In  short,, 
it  takes  a  myriad  of  things  to  go  right  for  a  program  to  be  successful,  but 
only  a  few  things  to  go  wrong  to  cause  major  problems. 

Weapon  system  issues  have  been  discussed  from  many  angles  and  in  a 
variety  of  categories  over  the  years.  Issues  such  as  cost  growth, 
concurrent  schedules,  reliability,  and  affordability  are  often  interrelated  in 
that  trying  to  resolve  one  problem  can  cause  or  exacerbate  others.  In  this 
chapter,  we  discuss  these  issues  within  the  broader  context  of  six 
questions,  which  should  be  asked  of  weapon  systems  as  they  proceed 
through  the  acquisition  cycle; 

•  Is  the  system  the  best  solution  to  the  mission  need? 

•  Are  the  program  cost  and  schedule  estimates  reasonable? 

•  Can  the  program  be  executed  with  avtiilable  funds? 

•  Is  the  program’s  .acquisition  strate^  reasonable? 

•  Has  it  been  shoM'n  that  the  system  is  operationally  effective? 

•  Is  the  system  suitable  for  production  and  fielding? 

Table  I.l,  in  appendix  I,  identifies  many  of  the  weapon  progranrs,  by  issue, 
on  which  we  have  reported  problems  over  the  past  15  years.  It  is  not  an 
all-inclusive  list  of  program  problems  but  is  presented  to  illustrate  how 
widespread  the  problems  have  been.  The  appendix  also  provides  detailed 
examples  on  a  number  of  programs. 

In  addressing  weapons  acquisition  problems,  we  have,  through  the  years, 
made  many  recommendations  to  the  Congress  and  pod.  While  these 
recommendations  have  had  an  impact,  the  persistence  and  pervasivene.ss 
of  the  problems  suggest  that  there  are  underlying  problems  that  are  not 
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being  addressed.  In  chapter  3  we  discuss  those  factors  we  believe  explain 
the  persistence  of  many  of  the  problems  discussed  below, 


s  the  System  the  Best 
lolution  to  the 
/lission  Need? 


DOD  acquisition  policies  require  analyses  of  mission  needs,  costs,  and 
alternatives  to  ensure  that  cost-effective  solutions  are  matched  to  valid 
needs  before  substantial  resources  are  committed  to  a  particular  program 
An  important  objective  is  to  minimize  overlap  and  duplication  among 
weapon  systems  that  perform  the  same  or  similar  missions.  This  is  of 
particular  concern  when  more  than  one  service  participates  in  similar 
mission  areas.  Generally  speaking,  our  work  has  concentrated  more 
heavily  on  whether  the  military  services  have  thoroughly  and  soundly 
analyzed  solutions  to  a  mission  need  thai\  it  has  on  whether  tl\e  need  is 
valid.  We  have  found  that,  while  the  ser  ices  conduct  considerable 
analyses  in  justifying  major  acquisitions,  these  analyses  can  be  narrowly 
focused,  not  fully  considering  alternative  solutions,  including  the  joint 
acquisition  of  systems  with  the  other  services. 


The  consideration  of  alternatives  to  the  Air  Force’s  $3.5  billion  Sensor 
Fuzed  Weapon  program  is  one  example  of  the  narrow  focus  of  some  of 
these  analyses  (see  fig.  2.1). 
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ure  2.1:  Insufficient  Examination  of  Alternatives 


Multiple  Launch  Rocket  System  Sensor  Fuzed  Weapon  System 

Army  weapon  systems,  such  as  the  Multiple  Launch  Rocket  System,  were  not  evaluated  by 
the  Air  Force  as  possible  alternatives  to  developing  the  Sensor  Fuzed  Weapon  System. 


In  an  August  1991  report,  we  discussed  the  Air  Force’s  plans  to  use  the 
Sensor  Fused  Weapon  primarily  to  interdict  enemy  follow-on  forces  before 
they  could  reinforce  or  replace  troops  at  the  front  lines.'  We  found  that  the 
Air  Force’s  cost  and  operational  effectiveness  analysis  had  not  considered 
the  full  range  of  weapons  available.  Weapons  such  as  Air  Force  mines  and 
Army  surface-to-surface  and  air-to-surface  missiles  and  rocket  systems 
were  excluded,  dod  said  the  Army  systems  had  noi  been  consider 'd 
appropriate  for  inclusion  in  the  analysis  because  each  service  had  a  valid, 
complementary  requirement  to  engage  enemy  targets  and  should  procure 
weapons  to  kill  those  targets.  We  believe  such  a  policy  enables  the 
services  to  pursue  parochial  solutions  regardless  of  what  the  other 
services  are  doing,  unnecessarily  increasing  Dcro’s  development, 
production,  and  support  costs. 


'  M  n  nitit  )?is  I  ’rt  icurci  i  oj^.s  I  Effort*  Proceeding  Wi*h  Senstjr  Weapon 

ProdiicUttri  ( ^rA(  VNSlAl Auj'.  l(i,  lliUl). 
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Similarly,  in  April  1992,  we  reported  on  dod’s  determination  of  weapon 
system  requirements  for  its  close  support  mission.^  Again,  we  found  that 
Air  Force  and  Army  analyses  of  alternatives  to  satisfy  their  mission  needs 
have  been  limited  to  specific  types  of  weapons  within  their  purview.  The 
analyses  gave  little,  if  any,  consideration  to  the  contributions  of  other 
close  support  weapons,  especially  those  from  another  service  branch.  This 
and  other  (jxamples  are  discussed  in  more  detciil  in  appendix  I. 


Axe  the  Program  Cost 
and  Schedule 
Estimates 
Reasonable? 


Cost  growth  and  schedule  delays,  two  of  the  most  prevalent  acquisition 
problems,  are  ^llso  among  the  oldest  and  most  visible  problems  associated 
with  weapon  systems.  In  March  1794,  Congress  authorized  the  building  of 
six  large  frigates,  which  were  to  form  the  backbone  of  the  Navy.  The 
then-War  Department  was  assigned  the  task  of  acquiring  the  slups.  Nearly 
17  months  later,  six  keels  were  laid.  Shortly  thereafter,  due  to  delays  and 
cost  overruns,  the  program  was  cut  back  to  three  frigates.  Today,  almost 
200  years  later,  dod  is  still  faced  with  tlie  same  problems:  it  takes  longer 
and  costs  more  to  develop  and  produce  weapons  than  the  estin.ates  on 
which  the  programs  were  initially  approved.  These  problems  not  only 
disrupt  the  program  at  hand;  they  cause  perturbations  in  other  programs 
as  well.  Furthermore,  their  occurrence  suggests  that  previous  program 
decisions  were  based  on  inaccurate  information. 


^Mayor  Acquisitions:  DOD’s  Process  Does  Not  Ensure  Proper  Mix  for  Clo.‘,e  Support  Mission 
(GAG7NS1AD-92-180,  Apr.  17, 1&92). 
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^  III  I 


e  2.2:  Significant  Cost  and  Schedule  Overruns 


Arleigh  Burke  DDG-51  Class  Destroyer 


The  cost  and  time  to  acquire  weapons,  such  as  these,  are  often  significantly  greater 
than  initial  estimates. 
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Not  ui\conunonly,  it  takes  10  to  15  years  to  design  and  develop  a  weapon 
system  and  to  produce  and  deploy  initial  operationally  capable  units. 
During  this  period,  weapon  systems,  for  reasons  both  within  and  outside 
the  control  of  the  program,  tend  to  experience  significant  cost  growth  and 
schedule  delays.  Program  cost  increases  on  the  order  of  20  to  40  percent 
have  been  common  on  major  weapon  programs,  with  numerous  programs 
experiencing  increases  much  greater  than  that.  These  increases  become 
more  telling  when  translated  into  unit  costs.  For  example,  the  estimated 
unit  cost  of  the  Comanche  helicopter  has  more  than  doubled  since 
May  1985.  Similarly,  schedule  delays  are  experienced  on  almost  every 
program,  with  the  accumulation  of  delays  on  some  adding  up  to  4  or  more 
years.  Table  2.1  summarizes  the  length  of  time  between  program  initiation 
and  fielding  of  initial  units  on  32  weapon  systems  in  production  and 
deployed  on  December  31, 1991. 


Table  2.1 :  Delays  In  Major  Weapon 
System  Programs 

Type  of  weapon 
(number  o1  programs) 

Average 

planned 

length 

(years) 

Average 
actual  length 
(years) 

Average 
total  delay 
(years) 

Average 

increase 

(percent) 

All  programs  (32) 

8.53 

10.49 

1.96 

22.94 

Aircraft  (11) 

8.36 

9.75 

1.39 

16.57 

Ground  vehicles  (3) 

6.75 

8.67 

1.92 

28.38 

Missiles  (10) 

8.97 

11,47 

2.50 

27.86 

Ships  (3) 

7.36 

9.17 

1.81 

24.57 

Other (5) 

9.75 

11,98 

2.23 

22.91 

Source:  GAO  analysis  of  DOD  Selected  Acquisition  Reports. 

The  extent  of  delays  may  be  even  greater  than  suggested  by  the  table.  The 
B-IB  program,  for  example,  reported  no  delays  in  meeting  its  initial 
fielding  date  of  September  1986.  However,  the  aircraft  was  deployed  with 
mission-critical  deficiencies  in  its  defensive  aviorucs  sy.stem,  which  has 
never  met  its  performance  requirements.  The  capability  of  the  aircraft 
continues,  to  this  day,  to  be  significantly  impaired.  In  addition,  programs 
suffer  delays  after  deployment  begins,  when  plarmed  production  rates  'are 
not  achieved  and  production  is  stretched  out  over  a  greater  number  of 
years  than  plarmed. 

Program  cost  increases  and  schedule  delays  are  often  the  manifestation  of 
other  program  problems.  For  example,  it  takes  additional  time  and  money 
to  accommodate  an  expansion  in  prugiaiu  scope,  to  overcome  {.echnical  or 
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production  problems,  and  to  restructure  a  program  to  absorb  funding 
reductions.  On  the  other  hand,  cost  and  schedule  problems  often  result 
from  flaws  within  the  estimates  themselves. 

The  desire  of  program  sponsors  to  keep  cost  estimates  as  low  as  possibl 
and  to  present  attractive  milestone  schedules  has  encouraged  the  use  of 
unreasonable  assumptions  about  the  pace  and  magnitude  of  the  technicj 
effort,  material  costs,  production  rates,  savings  from  competition,  and 
other  factors.  In  some  cases,  acquisition  cost  estimates  have  been  kept 
low  by  excluding  relevant  program  costs — ^such  as  the  cost  of  training 
equipment — ^which  should  be  included  in  program  cost  estimates. 
Moreover,  cost  and  schedule  estimates  are  interdependent.  A  schedule 
delay,  assuming  program  scope  is  not  reduced,  Avill  likely  drive  program 
cost  up.  Similarly,  a  cost  increase  will  likely  protract  a  program  schedule 
unless  more  money  becomv^s  available. 

We  have  reported  on  cost  and  schedule  problems  many  times  over  the  la 
15  years  (see  app.  I).  Gur  September  1991  report  on  missile  acquisition 
programs  provides  a  good  synopsis  of  these  problems.^  We  reviewed  the 
DOD  missile  systems  in  production  that  had  at  least  5  yeans  of  production 
experience.  Each  had  encountered  cost  and  schedule  overruns,  with  the 
unit  acquisition  cost  for  9  of  the  12  having  increased  20  percent  or  more 
over  the  planning  estimates.  A  detailed  examination  of  eight  systems 
found  that  the  unit  cost  and  schedule  planning  estimates  were  often  ove; 
optimistic,  not  adequately  reflecting  the  risks  associated  with  the  missik 
system’s  design,  development,  and  production.  Costs  grew  and  delays 
occurred,  reflecting  the  increcised  technological  development  required  a 
the  greater-than-anticipated  complexity  of  the  production  processes.  Ke; 
DOD  program  reviews,  designed  to  help  ensure  that  service  cost  and 
schedule  estimates  were  not  overly  optimistic,  did  not  sufficiently  assesi 
the  technical  assumptions  behind  the  estimates. 


n  the  Program  Be 
ecuted  With 
ailable  Funds? 


Understated  program  cost  estimates  are  a  mqjor  contributor  to  the 
broader  problem  of  affordability,  which,  in  simple  terms,  can  be  stated  a 
“too  many  weapon  systems  chasing  too  few  dollars."  dod’s  tendency  to 
overestimate  the  amount  of  future  funding  available  for  defense,  coupIe< 
with  the  tendency  to  underestimate  program  costs,  has  resulted  in  more 
programs  being  started  than  can  be  executed  intact  (see  fig.  2.3). 


^Tactical  Missile  Acquisitions:  Understated  Technical  Risks  Leading  to  Cost  and  Schedule  Overruns 
(GAO/NSIAD-ni-280,  Sept  17, 1991); 
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,ure  2.3;  Questionable  Affordability 


Seawolt  SSN-21  Attack  Submarine  Program  8-2  Advanced  Technology  Bomber  Program 


Concerns  regarding  affordabiiity  have  been  raised  on  these  and  other  weapon  programs. 


The  basic  framework  in  which  dod  delermines  the  affordability  cf 
programs  is  the  Future  Years  Defense  Program.  It  is  a  6-year  projection  of 
available  funds  (formerly  a  5-year  projection)  allocated  to  individual 
programs.  Because  all  projected  funds  are  allocated  to  programs,  any 
significant  increase  in  the  demand  for  funds  or  any  decrease  in  available 
funds  creates  a  funding  shortfall.  kYinding  shortfalls,  whether  stemming 
from  across-the-board  cuts  in  defense  spending,  the  unanticipated  start  of 
a  new  program,  or  cost  growth,  are  eventually  resolved  at  the  individual 
program  level.  Most  often,  programs  are  stretched  and  reshuffled  to  fit 
within  lower  annual  funding  levels  than  planned — with  the  side  effects 
frequently  being  unit  cost  increases,  schedule  delays,  and  program 
instability.  Figure  2.4  illustrates  the  gap  between  actual  and  projected 
funds,  as  well  as  progress  non  has  made  regarding  funding  projections. 
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ure  2.4:  Closing  the  Gap  Between  DOD’s  Program  and  Budget  Realities 
>  Dollaft  In  billions 


Fiscal  yssrs 


Future  Years  Defense  Program 
■■  Actual  obllgatlonal  authority 
•  ••••■  Estimated  budget  authority 


Source:  GAO  analysis  of  DOD  budget  data. 


Evident  in  the  figure  is  the  large  mismatch  between  the  funding  levels  in 
dod’s  program  plmis  and  those  funds  actually  or  likely  to  be  appropriated. 
For  ex£imple,  a  shortfall  of  almost  $553  billion  existed  between  the  1986-90 
program  plan  and  the  funds  actually  appropriated.  Equally  evident  Is  the 
fact  that  dod’s  program  plans  have  become  more  reasonable.  Since  the 
1986-90  plan,  the  inismatch  between  planned  funds  and  likely 
approj'riations  has  shrunk  steadily  and  was  nearly  eliminated  altogether 
by  DOD  in  its  latest  plans. 

It  is  clear  that  dod  has  taken  some  steps  to  address  the  affordability 
problem — and  that  these  steps  have  been  accentuated  by  the  firm  full 
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funding  policy  enunciated  in  1991  by  top  DOD  acquisition  officials. 
However,  the  problem  is  not  yet  resolved.  The  requirements  and 
acquisition  processes  continue  to  allow  programs  to  proceed  without  fully 
addressing  affordability.  Hard  decisions  regarding  program  delays  or 
terminations  are  often  put  off  to  the  budget  preparation  process,  when 
program  plans  must  fit  within  the  President’s  budget  request.  Although 
DOD  may  be  identifying  potential  offsets  under  the  new  policy,  we  have 
found  that  actual  offsets,  as  in  the  past,  are  not  being  identified  eind  made 
until  later,  as  part  of  dod’s  normal  budget  process.^ 

Further,  an  important  factor  in  the  affordability  equation  not  captured  in 
figure  2.4  is  the  widening  effect  that  unplanned  cost  growth  in  weapon 
programs  has  on  the  funding  mismatch.  That  is,  if  program  costs  were 
reasonably  estimated  and  the  pace  and  quantities  called  for  in  the 
individual  progrsim  plans  were  not  changed,  the  demand  for  funds  woulc. 
actually  exceed  the  levels  currently  projected  in  dod’s  program  plans. 
Thus,  cost  growth  will  still  provoke  an  affordability  problem,  even  if 
funding  projections  are  reasonable. 


[s  the  Program’s 
Acquisition  Strategy 
Reasonable? 


Two  divergent  criticisms  are  commonly  levied  at  weapon  acquisition  plans 
or  strategies.  First,  weapons  take  too  long  to  develop  and  field;  second, 
weapons  begin  production  too  quickly  and  are  fielded  with  m^yor 
unknowns  or  unresolved  problems.  Together,  these  criticisms  mirror  the 
tension  among  competing  demands  that  an  acquisition  strategy  must 
attempt  to  satisfy. 

The  acquisition  strategy,  which  is  a  comprehensive  plan  of  how  to  achieve 
the  weapon  system  program’s  goals  and  objectives,  is  a  m^yor  determinant 
of  program  outcomes.  It  is  the  service’s  plan  for  developing,  fielding,  and 
supporting  a  weapon,  including  the  managerial,  technical,  and  contractual 
approaches.  A  key  element  of  the  strategy  is  the  program  schedule,  which 
is  pimctuated  by  major  events  such  as  testing  and  key  decision  points.  The 
ty^o  most  basic  demands  an  acquisition  strategy  must  meet  are  inherently 
conflicting — developing  and  fielding  the  weapon  as  quickly  as  possible  to 
counter  the  threat,  while  minimizing  technical  and  cost  risks.  A  strategy 
optimized  for  accelerated  fielding  will  likely  accept  higher  risk  primarily 
through  concurrent  development  and  production.  Under  such  a  strategy, 
major  problems  are  more  likely  to  be  discovered  in  production,  when  it  is 
either  too  late  or  very  costly  to  correct  them.  On  the  other  hand,  a  strategy 


W/eapons  Acquisition:  Implementation  of  the  1091  DOD  b>jll  blinding  Policy  (GAO/N  SI  AD-92-238, 
Sept.  24, 1992). 
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optimized  for  risk  aversion  will  result  in  a  prolonged  development 
schedule  and  increased  developmental  costs. 

Most  acquisition  strategies  compromise  between  these  and  other 
demands,  such  as  the  technical  challenge  being  attempted  and  the  amou 
of  funding  available.  We  have  found  that  in  striking  these  compromises, 
acquisition  strategies  embody  optimistic  assumptions  regarding  the 
difficulty  of  the  technical  effort;  the  outcome  and  timing  of  key  events, 
such  as  testing;  eind,  as  discussed  earlier,  the  cost,  schedule,  and 
affordability  of  the  effort.  The  inevitable  result  has  been  acquisition 
strategies  that  are  tightly  strung,  being  both  sensitive  and  susceptible  t  o 
perturbations  such  as  funding  reductions  and  unanticipated  technical 
problems.  Apart  from  causing  cost  increases,  such  problems  result  in  tin 
investment  of  additional  time  and  effort,  the  acceptance  of  lower 
performance,  or  the  acceptance  of  higher  risk,  either  in  the  form  of 
increased  concurrency  or  a  reduction  in  the  scope  of  the  development 
effort. 


Perhaps  the  most  troublesome  characteristic  of  acquisition  strategies  in 
the  1970s  and  1980s  was  the  high  degree  of  concurrency  between  the 
development  zmd  production  of  weapons.  “Concurrency”  can  be  broadlj 
defined  as  the  practice  of  beginning  production  before  completing 
development,  testing,  and  evaluation.  Concurrency  can  be  used  to 
expedite  the  acquisition  and  deployment  of  weapon  systems,  and  a  cerU 
amount  of  it  can  make  good  management  sense.  For  example,  proving  o 
critical  production  technologies  in  development  can  avert  major  probler 
in  production.  However,  the  reason  most  commonly  cited  for  using  a 
concurrent  acquisition  strategy  has  been  to  expedite  development  and 
production  so  the  weapon  can  be  fielded  quickly  to  counter  the  Soviet 
threat.  Concurrency  is  also  used  to  absorb  delays  caused  by  cost,  fundir 
technical,  or  other  problems.  Such  an  approach  increases  program  risk, 
particularly  when  complex  or  novel  technologies  are  involved. 
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Advanced  Medium  Range  Air-to-Air  Missile  Sergeant  York  Air  Defense  Gun 


Beginning  production  before  reducing  program  risks  has  led  to  costly  problems  on  these 
and  other  weapon  systems. 


At  the  very  least,  a  highly  concurrent  strategy  forces  decisionmakers  to 
make  key  decisions  without  adequate  information  about  the  weapon’s 
demonstrated  ojicrational  effectiveness,  reliability,  logistic  supportabiliiy, 
and  readiness  for  production.  Also,  rushing  into  production  before  critical 
tests  have  been  successfully  completed  has  resulted  in  tlu*  piuxdiase  of 
systems  that  do  not  perform  ;is  intended.  These  premature  purchases  have 
affected  the  oju’ rational  readiness  of  our  forces  and  have  quite  often  led  to 
expensive  modifi(;ations.  Among  the  most  celebrated  examples  of 
excessive  concurrency  are  t  he  C-5A  cargo  aircraft  and  the  B-IB  bomber 
lirograms.  The  C-.5A  enteri'd  production  before  the  aircraft  was  fully 
tested,  which  led  to  a  12-year  wing  modification  program  costing  about 
$1 .13  billion  to  correct  problems.  On  the  B-IB,  full-scale  diwelopment  and 
Ifroduction  contracts  wi're  awarded  on  the  sanu'  day  for  thc'  aircraft  s 
dehmsive  avionics  system,  which  has  sin<;e  h(u*n  plagued  with  problems 
that  have  sr'i’iou.sly  imjiaired  the  aircraft's  (.'aijability. 
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When  weapon  system  development  and  production  schedules  become 
more  concurrent  than  planned,  Jie  critical  function  of  independent 
operational  test  and  evaluation  often  suffers.’’  Such  tests  are  crucial  for 
assessing  mission  performance  before  making  significant  program  dollar 
commitments.  In  May  1990,  we  reported  that,  based  on  our  review  of  six 
weapon  systems  and  other  audit  work,  operational  test  eind  evaluation 
results  often  were  not  available  to  support  decisions  to  start  production 
because  the  military  services  failed  to  plan  for  such  testing.®  In  June  1985, 
we  reported  on  the  testing  and  evaluation  of  five  weapon  systems — the 
Air-Launched  Cruise  Missile,  the  B-IB  bomber,  the  Sergeant  York  air 
defense  gun,  the  F/A-18  aircraft,  and  the  AGM-88A  High  Speed 
Anti-Radiation  Missile.^  We  disclosed  that  dod  had  not  obtained  operatiom 
test  results  on  any  of  the  five  systems  before  beginning  production.  On 
four  of  the  five  weapons  we  identified  negative  effects,  including 
expensive  retrofits  or  modifications.  The  Sergeant  York  program 
demonstrated  the  most  extreme  cor^sequence.  After  the  Army  had  spent 
$2  billion  and  produced  64  of  the  614  gun  systems,  the  Secretary  of 
Defense  tenmnated  vhe  program  because  operational  tests  showed  that 
the  system  was  only  a  marginal  improvement  over  the  existing  air  defense 
system. 


Even  if  the  operational  test  and  evaluation  is  timely,  methodological 
shortcomings  can  inhibit  its  effectiveness.  Common  weaknesses  in  the 
quality  of  such  testing  that  we  have  reported  include  the  lack  of  realism, 
independence,  and  test  resources  in  the  planning,  execution,  and 
evaluation  of  the  tests.  We  have  also  reported  on  long-standing  problems 
with  the  completeness  and  accuracy  of  test  and  evaluation  reports 
provided  by  the  services  to  the  Office  of  the  Secretary  of  Defense  and 
Congress. 

In  commenting  on  a  draft  of  this  report,  dod  officials  said  they  are  now 
lessening  the  amount  of  concurrency  between  development  and 
production  in  weapon  programs.  This  is  possible,  they  said,  because  of  the 
end  of  the  Cold  War. 


^Shortof  war,  operational  testing  is  the  most  realistic  way  of  assessing  weapon  system  performance.  1 
puLs  a  weaixm  through  the  rigfirs  of  combat  conditions  to  determine  its  operational  effectiveness  and 
suitability. 

^^Weapons  Testing:  DOD  Needs  to  Plan  and  Conduct  More  Timely  Operational  Tests  and  Kvaluatiori 
May 

^P^oduction  of  Some  Majfir  Weapon  Systems  Began  With  Only  Limited  Operational  Test  and 
h>alu^i^i  Rebults  (GA0/NS1AD~85-68,  June  19, 1985). 
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Although  DOD  has  produced  many  of  the  best  and  most  advanced  weapons 
in  the  world,  these  weapons  often  encountered  significant  performance 
problems  during  them  acquisition  (see  fig.  2.6).  In  the  extreme,  some 
weapons — like  the  Aquila  Remote  Piloted  Vehicle  and  the  Rolling  Airframe 
Missile — were  unable  to  perform  their  missions  emd  were  either  canceled 
or  underwent  costly  improvements.  The  more  common  problem,  however, 
occurs  when  testing  or  field  experience  shows  that  a  weapon  falls  short  of 
meeting  all  its  performance  requirements,  even  though  it  may  offer  better 
performance  than  its  predecessor.  In  these  instances,  redesigns  or 
upgrades  are  required  to  achieve  desired  performance,  most  often  at  the 
expense  of  increased  program  costs  and  of  delays  in  the  fielding  of  the 
weapon.  In  some  instances  the  shortfalls  are  accepted. 
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Rolling  Airframe  Missile 


The  ability  of  these  and  otfier  systems  to  perform  effectively  has  been  a  persistent  acquisition  issue. 


Successful  perfomiance  of  missions,  assuming  well  trained  operators  and 
reliable  equipment,  requires  a  weapon  system  that  is  operationally 
effective.  '*Effc  iveness”  generally  refers  to  the  ability  of  a  weapon  to 


Page  29 


GAO/N,SlAD«93"l5  Lasting  Change  in  Weapons  Acqut»itio 


Chapter  2 

Weapon  Acquiaitiona  Experience  a 
Persistent  Set  of  Problems 


successfully  engage  the  enemy.  For  a  meyority  of  weapons,  that  means  the 
ability  to  reach  the  target  area,  find  targets,  and  destroy  or  disable  them 
with  munitions  such  as  missiles  or  torpedoes.  An  important  element  of 
effectiveness  is  “survivability,”  or  the  capacity  of  a  weapon  to  withstand 
the  effects  of  an  enemy’s  firepower  and  countermeasures.  Ultimately,  the 
yardstick  for  measuring  effectiveness  is  the  capability  of  the  threat.  Thus, 
even  if  a  weapon  meets  its  requirements,  an  advance  In  the  threat  could 
impose  sigtuficant  performance  problems. 

Given  the  complexities  and  uncertainties  in  weapon  development  and 
testing,  the  identification  of  performance  problems  is  not  unusual.  In  fact, 
one  of  the  main  objectives  of  the  acquisition  process  is  to  identify  and 
correct  such  problems  early.  Debate  and  controversy  most  often  arise  over 
the  seriousness  of  the  problem  and  the  proper  corrective  action.  With 
continual  advances  in  the  threat  emd  design  philosophies  that  promote  the 
incorporation  of  new  and  emerging  technologies  in  weapons,  the 
challenge  of  overcoming  problems  has  been  continual.  An  area  of 
immense  technological  growth  has  been  computer  software  for  weapon 
systems.  For  example,  the  Air  Force’s  Vietnam-era  F-4  fighter  had 
practically  no  softwai’e;  the  current  F-16  requires  236,000  lines  of  software 
code;  and  the  developmental  F-22  is  expected  to  require  7  million  lines  of 
software  code.  The  problems  that  accompany  the  development  of  software 
have  led  to  the  emergence  of  software  as  a  formidable  challenge  in 
weapon  system  development  euid  testing.  In  fact,  software  has  been 
described  as  the  “Achilles  heel”  of  modern  weapon  systems  because  it  is 
often  a  key  determinant  of  development  schedules  and  because  the 
perfonnance  of  key  functions  such  as  navigation,  enemy  detection,  and 
fire  control  depends  on  it. 


s  the  System  Suitable 
br  Production  and 
ridding? 


Although  operational  effectiveness  problems  often  attract  the  most 
attention,  we  have  found  that  many  weapons  encounter  significant 
problems  on  the  production  line  and  in  the  field.  It  is  not  uncommon  for 
the  actual  production  of  a  weapon  to  require  much  greater  effort  than 
anticipated,  with  resulting  cost  and  schedule  increases.  Similarly,  we  have 
frequently  found  that  weapons  break  down  more  often  and  are  more 
difficult  to  logistically  support  than  planned.  Apart  from  the  cost  impact, 
such  we^dcnesses  in  the  operational  suitability  of  weapons  can  impair  their 
readiness  and  ability  to  carry  out  missions.  As  with  operational 
effectiveness,  the  timely  identification  of  operational  suitability  problems 
depends  on  the  quality  of  test  and  evaluation;  the  avoidance  of  such 
problenrs  depends  on  a  sound  design  proces.s. 
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duction  Readiness  it  is  dod’s  policy  to  begin  planning  for  production  early  in  the  acquisition 

process  to  ensure  that  the  weapon  system  design  not  only  meets 
performance  objectives  but  also  can  be  produced  in  an  economical  and 
timely  manner.  Experience,  however,  has  shown  that  new  weapon  systei 
frequently  encounter  great  difficulties  as  they  begin  production  (see  fig. 
2.7).  Problems  on  the  production  floor  commonly  result  in  high  unit 
production  costs,  late  deliveries,  high  maintenance  demands,  and  poor 
field  reliability.  Production  cost  increases  on  the  order  of  50  percent  are 
not  unusual  and  can  greatly  disrupt  funding  plans,  schedules,  and  progra 
quantities. 


erational  Suitability 
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In  a  May  1985  report,  we  analyzed  the  experience  of  six  weapon  systems 
as  they  made  the  transition  from  development  to  production.®  We  found 
that,  in  varying  degrees,  production  preparations,  such  as  producibility 
studies  and  manufacturing  technology  projects,  for  four  of  the 
programs — the  Copperhead  projectile.  Black  Hawk  helicopter.  Tomahawk 
cruise  missile,  and  High  Speed  Anti-Radiation  Missile  programs — had  been 
sporadic  and  underfunded  and  had  been  largely  compressed  into  the  late 
stages  of  development  and  early  stages  of  production.  Program  success 
was  equated  with  techiucal  performance,  even  at  the  time  of  production 
decisions.  As  a  result,  these  weapons  encountered  significant  difficulties 
in  production  that  resulted  in  increased  costs  and  delayed  deliveries.  On 
the  other  hand,  production  preparations  for  the  F-16  fighter  and  the 
Air-Launched  Cruise  Missile  had  been  thorough  and  timely  because 
technical  and  production  concerns  had  received  balanced  treatment. 
Consequently,  these  two  weapon  systems  entered  production  vrithout 
delay  and  mgjor  cost  increases. 

Despite  increased  recognition  by  dod  during  the  1980s  of  the  importance 
of  addressing  producibility  in  the  acquisition  process,  we  have  continued 
to  witness  production  problems  on  some  of  the  very  latest  acquisitions, 
including  the  B-2  bomber,  the  SSN-21  attack  submarine,  and  the  Advanced 
Cruise  Missile. 


The  technology  that  has  made  high-performance  weapons  possible  has 
also  introduced  new  challenges  to  weapon  system  designers  to  make  these 
weapons  suitable  for  field  operations.  To  be  operationally  suitable, 
weapons  must,  among  other  things,  be  able  to  be  effectively  operated, 
maintained,  and  supported  by  the  military  forces.  Our  reviews  have 
disclosed  that  design  considerations  such  as  reliability,  maintainability, 
and  logistics  support  ha/e  been  compromised  or  otherwise  not  adequately 
coirsidered  during  the  acquisition  process  (see  fig.  2.8).  Performance  and 
schedule  requii  ements  tend  to  take  precedence  over  operational 
suitability  concerns,  particulaily  when  funding  shortfalls  force  trade-offs. 
The  result  has  often  been  very  high  maintenance  and  support  costs  and 
lower-than-expected  availability  for  operations. 


^Why  Some  Weapon  System?  Encounter  l^oducUon  Problems  While  Othei*s  Do  Not:  Six  Case  Studies 
(GAO/NSIAD-85-34,  May  24,  1985). 
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igure  2.8:  Operational  Support  Problems 


M-1  Main  Battle  Tank 


The  maintenance  and  support  required  of  many  high  performance  weapon  systems,  such  as 
these  two  Army  systems,  are  often  very  costly  and  can  affect  the  system's  operational  availability. 

Limitations  in  the  quality  of  test  and  evaluation — such  as  the  immaturity  c 
prototj'pes  and  optimistic  interpretation  of  reliability  data — can  obscure 
or  delay  recognition  of  operational  suitability  problems.  For  example,  in 
reviewing  the  Army’s  evaluation  of  operational  tests  completed  in  1990  or 
its  Air  Defense  Antitank  System,  w^e  found  that  the  Army  had  excluded  th 
test  data  from  missile  firings  in  calculating  the  system’s  operational 
availability.®  These  firings  were  critical  because  they  marked  the  only  tim 
during  testing  that  the  entire  system  was  in  operation.  In  addition, 
although  the  system  requirements  stipulated  that  all  unscheduled 
maintenance  tinre  be  included  in  test  calculations,  we  identified  at  least 
145  hours  of  such  maintenance  that  had  not  been  included.  The  effect  of 
these  exclusions  was  to  overstate  the  system’s  demonstrated  operational 
availability. 


"Army  A('()ui.sili(m:  Air  Defense  Antitank  System  Did  Not  Meet  Operational  Test  Objeetives 
CGAO/NSIAD-91-5I,  Dec.  10,  1090). 
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Although  DOD  took  steps  during  the  1980s  to  place  increased  emphasis  on 
operational  suitability  considerations  during  the  acquisition  process,  we 
continue  to  witness  weapon  systems  being  deployed  without  reliable 
support  and  test  equipment  or  with  design  problems  that  require  retrofits 
and  modifications  to  make  them  suitable  for  field  use.  Examples  include 
the  Apache  helicopter,  the  Advanced  Medium  Range  Air-to-Air  missile,  and 
various  electronic  warfare  systems,  including  their  test  equipment. 
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onsequences  of  the  Acquisition  Culture 


While  occurrences  such  as  perform^lnce  shortfalls,  schedule  delays,  and 
cost  increases  are  recognized  as  persistent  problems  in  weapon 
acquisitions,  we  believe  that  they  should  also  be  viewed  as  the  logical 
consequences  of  the  acquisition  culture.  This  culture  can  be  defined  as  th( 
collective  behavior  of  the  various  participants  in  the  acquisition 
process — principally  dod  and  the  Congress — ^and  the  forces  that  motivate 
their  behavior.  In  fact,  the  process  may  be  more  realistically  portrayed  as 
the  interaction  of  its  participants  than  as  the  methodological  procedure 
depicted  on  paper.  This  culture  has  evolved  as  the  acquisition  process  has 
become  a  vehicle  for  meeting  the  diverse  needs  of  its  participants  through 
the  steady  initiation  and  sustainmem  of  programs. 

While  individual  participants  see  their  needs  as  rational  and  aligned  with 
the  national  interest,  collectively,  these  needs  create  incentives  for 
pushing  programs  and  encouraging  undue  optimism,  parochialism,  and 
other  compromises  of  good  judgment.  Under  these  circumstances, 
persistent  performance  problems,  cost  growth,  schedule  slippage,  and 
difficulties  with  production  and  field  support  carmot  all  be  attributed  to 
errors,  lack  of  expertise,  or  unforeseeable  events.  Rather,  a  level  of  these 
problems  is  embedded  as  the  undesirable,  but  apparently  acceptable, 
consequence  of  the  process.  These  problem '  persist  not  because  they  are 
overlooked  or  under-regulated,  but  because  they  enable  more  programs  t< 
survive  and  thus  more  needs  to  be  met.  The  problems  are  not  the  fault  of 
program  managers  or  any  single  pairticipant;  they  are  the  collective 
responsibility  of  all  participants. 


rhe  Acquisition 
F'rocess  Responds  to 
:he  Needs  of  Its 
Participants 


Witlun  die  framework  of  national  security,  the  acquisition  process 
generally,  and  weapon  programs  specifically,  must  make  winners  of  the 
key  participants.  On  this  score,  the  acquisition  process  does  quite  well.  A: 
a  weapon  makes  the  difficult  journey  from  concept  to  the  field,  it  is 
debated,  stretched,  threatened,  accelerated,  delayed,  and  redefined  to 
conform  to  the  needs  of  the  participants.  Even  critics  and  independent 
agencies  benefit  in  that  weapon  systems  provide  a  forum  for  debate  that 
helps  reinforce  their  roles.  Over  time,  the  nearly  unassailable  urgency  of 
the  threat  and  the  ability  of  weapon  programs  to  help  participants  achievi 
goals  have  institutionalized  a  culture  that  prefers  continuing  a  program  tc 
terminating  it  and  hinders  making  difficult  trade-offs  to  alleviate  cost, 
affordability,  duplication,  risk,  m\d  logistic  supportability  concerns. 
Collectively,  these  needs  shape  the  cultural  problem,  for  they  cannot  be 
easily  met  without  an  ample  supply  of  prograims. 


Page  35 


GAO/NSIAD-93-16  Lasting  Change  in  Weapons  Acquisiti 


Consequences  of  tiie  AcquisiUcn  Culture 


on  Programs 
(Sent  a  Common 
on  to  Diverse  Needs 


Weapon  systems  are  one  of  the  few  common  vehicles  for  meeting  a  wide 
variety  of  deferise  needs.  Witliin  the  acquisition  process  itself,  the  survival 
and  replenishment  of  weapon  programs  have  become  intertwined  with  the 
basic  needs  of  the  participants.  The  most  pervasive  need  over  the  last 
40  years  has  been  to  counter  the  threat  posed  by  the  Soviet  Union  and  the 
Warsaw  Pact.  Until  recent  years,  this  threat  was  seen  as  an  ever  advancing 
and  numerically  superior  force  that  necessitated  urgent  and  continuous 
U.S.  efforts  to  field  technologically  superior  weapons.  In  the  ly70s  and 
1980s,  a  general  consensus  developed  between  the  different 
administrations  and  the  Congress  on  the  need  to  modernize  U.S.  armed 
forces  through  new  weaponry.  This  consensus  was  manifested  by  the 
broad  range  of  programs  started  during  that  time. 

However,  the  process  of  acquiring  new  weapons — ^as  defined  by  the 
process’s  many  participants — ^is  far  more  complex  than  the  seemingly 
straightforward  purchase  of  equipment  to  defeat  an  enemy  threat. 
Acquisition  process  paiticipants  include  the  military  services  and  the 
individual  warfare  communities  within  each  service;  the  offices  of  the 
President,  the  Secretary  of  Defense,  and  the  Joint  Chiefs  of  Staff; 
independent  oversight  organizations;  contractors;  political  action 
committees;  professional  associations;  and  the  Congress.  Their  interaction 
often  reflects  a  conflict  between  the  optimism  that  accompanies  program 
sponsorship  and  the  skepticism  that  accompaiues  oversight  of  the  process 
Ironically,  the  presence  of  one  fuels  the  other.  While  program  sponsors  arc 
often  considered  synonymous  with  program  managers,  sponsors  include 
all  participants  who  support  a  program  on  its  own  merits  or  because  they 
stand  to  benefit  from  such  support.  On  the  other  hand,  participants  who 
challenge  or  oppose  individual  programs  are  considered  program  skeptics: 
these  can  include  independent  evaluators,  allocators  of  resources,  and 
supporters  of  competing  programs. 

Individual  needs  that  the  acquisition  process  attempts  to  satisfy  are 
depicted  in  figure  3,1. 
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Figure  3.1:  Weapon  Acquisition  Programs  Meet  Many  Diverse  Needs 


ficult  Choices  Carry 
V  Rewards 


Persistent  Problems  May  Be  the 
Consequences  of  the  Acquisition  Culture 


Although  individual  participants  may  act  rationally  based  on  their  views  of 
national  security  interests,  their  actions  are  governed  by  their 
backgrounds,  the  biases  of  their  institutions,  and  their  perceptions  of 
rew2U"ds  and  punishments.  Furthermore,  the  short  tenures  of  many 
acquisition  participants  encourage  them  to  take  actions  that  have 
near-term  payoffs.  More  near-term  rewards  can  be  fovmd  in  starting  and 
supporting  programs  than  in  making  difficult  decisions  in  the  present  to 
solve  longer  term  problems.  Participants  are  also  influenced  by  how  they 
anticipate  others  may  act;  for  example,  a  service’s  candor  about  test 
results  is  affected  by  how  these  results  could  be  used  by  program  critics. 

Collectively,  as  participants’  needs  are  translated  into  actions  on  weapon 
progreims,  the  purpose  of  such  programs  transcends  efficiently  filling  voids 
in  military  capability.  Thus,  weapons  have  become  integral  to  policy 
decisior«,  definitions  of  roles  and  functions,  justifioafions  of  budget  levels 
and  shau’es,  service  reputations,  influence  of  oversight  virgaruzations, 
distribution  of  funds  to  localities,  and  the  industrial  bast.  Programs  are 
also  important  to  individual  careers.  For  example,  a  program  manager’s 
success  depends  on  getting  results,  and  in  acquisitions,  results  mean 
getting  the  program  through  the  next  major  milestone  and  into  the  field. 
'Thus,  a  program  manager’s  strongest  motivation  is  to  keep  the  program 
moving  and  to  protect  it  from  interruption. 

Strong  incentives  for  supporting  programs  permeate  otlier  levels  of  the 
acquisition  process  as  well.  To  a  contractor,  the  basic  incentive  may  be 
winning  business.  To  a  service  br^lnch,  it  may  be  perpetuating  a  mission. 
To  a  service,  it  may  be  securing  its  reputation  and  its  share  of  the  budget. 
To  a  Member  of  Congress,  it  may  be  responding  to  constituency  interests. 
To  service  executives,  whose  tenures  are  often  relatively  short,  weapon 
systems  can  be  an  effective  way  to  leave  a  legacy.  Under  this  set  of 
incentives,  programs — like  the  B-1 — that  are  eventually  fielded  despite 
performance,  cost,  and  other  significant  problems  are  often  considered 
more  successful  than  programs — like  the  Sergeant  York — ^that  are 
canceled  because  serious  flaws  were  discovered  before  fielding.  When  in 
conflict  with  responsible  management,  “successful”  outcomes  may  carry 
more  rewards. 


While  decisions  to  create  and  sustain  acquisition  programs  are  encouraged 
by  strong  incentives,  decisions  that  restrict  or  control  programs  operate 
under  weak  iiicenlivet.,  Uiifortunately,  these  are  often  the  kinds  of 
decisions  necessary  to  solve  problems  and  to  improve  acquisition 
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management.  For  example,  because  funding  acquisitions  is  a  “zero  sum” 
exercise — that  is,  the  decisions  made  must  fall  \vithin  a  given  level  of 
spending — ^actions  such  as  improving  the  realisni  of  a  program’s  cost 
estimate  or  reducing  its  concurrency  will  either  jeopardize  the  program  al 
hand  or  take  resources  from  other  programs.  Such  decisions  face  strong 
disincentives  because  they  conflict  with  the  other,  more  powerful,  needs 
served  by  the  programs. 

The  traditional  difficulty  acquisition  process  participants  have  had  in 
dealing  squarely  with  affordability  limitations  is  a  case  in  point.  In  a 
collective  process  that  favors  compromise,  decision-makers  have 
preferred  to  find  ways  to  eifford,  rather  than  decide  whether  to  afford, 
individual  programs.  The  result  has  been  to  sustain  more  programs  at 
lower  funding  levels  rather  than  to  fully  fund  fewer  programs.  Under  the 
prevailing  culture,  program  sponsors  are  motivated  to  make  optimistic 
cost  assumptions  and  to  reduce  quantities  or  program  scope  or  to  prolonj 
the  schedule  to  make  a  program  affordable  and  thereby  avoid  cancellatioi 
Although  these  actions  do  not  solve  tlie  long-term  affordability  problem, 
they  lessen  the  need  for  decision-makers  to  make  difficult  choices  among 
programs.  Ultimately,  program  survival  has  proven  a  more  powerful 
incentive  than  program  affordability. 

Even  when  dod  makes  the  difficult  choice  of  canceling  a  program,  the 
Congress,  in  some  instances,  continues  to  support  it.  Such  was  the  case 
during  the  1992  budget  process,  when  dod  decided  to  discontinue  the  M-1 
tank  modernization,  V-22  Osprey  aircraft,  and  SSN-21  Seawolf  submarine 
progranris.  Each  of  these  programs  continued  to  receive  support  by  key 
congressional  committees,  and  dod  eventually  withdrew  its  opposition.  Ir 
other  instances,  new  weapon  systems  end  up  complementing,  rather  thai 
replacing,  existing  weapons  that  have  strong  constituencies.  For  example 
production  of  the  B-IA  strategic  bomber  was  terminated  in  1977  with  the 
advent  of  the  cruise  missile.  Although  the  cruise  missile  began  fielding  in 
1981,  the  bomber  program  was  resurrected  and  fielded  in  the  form  of  the 
B-iB. 


Thus,  although  the  acquisition  process  has  long  been  criticized  for  the 
instability  it  imparts  to  individual  programs,  the  process  may  in  fact  l  iave 
achieved  stabihty  at  a  more  basic  level.  Perhaps  reflecting  the  incentives 
for  preserving  programs,  the  number  of  programs  has  remained  relativelj 
stable  compared  with  the  growth  and  decline  in  fundmg  levels  over  the 
last  10  years,  as  shown  in  figure  3.2. 
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ire  3.2:  Weapon  System  Funding  Levels  and  Line  Items  Over  Time 

PsrcsnUgs  change  sirtea  fiscal  yaar  1976 


977  197$  1979  1980  1981  1962  1983  1984  1985  1988  1987  1988  1989  1990  1991  1992  1993 

Iscal  year 


mmm^m  Numbof  o(  Ilno  Itsms 
^  ^  Procurentent  funding  levels 


Notes:  Funding  levels  are  standardized  as  constant  fiscal  year  1992  dollars.  Data  for  fiscal  years 
1992  and  1993  are  estimates. 

Source:  GAO  analysis  of  DOD  budget  data. 


This  relative  steadiness  in  program  volume  suggests  a  broader  view  of 
program  instability  that  recognizes  tlie  incentives  of  the  acquisition 
culture.  It  is  true  that  programs  are  routinely  disrupted  as  a  result  of  dod 
and  congressional  actions,  prompting  the  charge  of  micromanagement.” 
On  the  other  hand,  this  same  propensity  for  disruption — ^versus 
caincellation — has  enabled  more  programs  to  survive  and  has 
accommodated  die  problems  that  tend  to  be  inherent  in  programs.  In  this 
sense,  individual  programs  are  not  the  innocent  victims  of  an  unstable 
acquisition  proce.ss;  tlieir  “instability”  may  well  have  been  the  price  paid 
for  survival. 
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Problems  Embedded 
at  the  Outset  of 
Programs 


Programs  Begin  as 
Parochial  Solutions 


Weapon  system  problems  can  often  be  traced  to  the  very  beginning  of  th 
acquisition  process.  In  addressing  mission  needs,  service  organizations 
propose  programs  that  perpetuate  their  existence  and  oversell  the 
programs  to  ensure  their  survival.  Predictably,  programs  begin  as 
parochial  solutions  embodying  highly  optimistic  estimates.  In  this  sense, 
program  justifications  may  be  more  heavily  influenced  by  the  participani 
needs  than  by  objective  decision-making.  That  is  not  to  say  that  the 
process  is  irrational  or  arbitrary.  To  the  contrary,  the  general  threat  may 
be  legitimate,  and  individual  program  analyses  may  be  objective.  Howev 
these  factors  do  not  necessarily  dominate  decision-making  because  they 
can  be  tailored  by  other  participants  in  the  process.  More  importantly,  if 
the  proposing  organizations  were  to  act  more  objectively,  they  could  do< 
their  own  programs  and  jeopardize  their  own  existence. 

The  decisions  made  in  justifying  a  program  cire  crucial  because  they 
largely  determine  the  eventual  performance,  schedule,  and  cost  of  the 
weapon  system.  An  approved  requirement  embodies  how  a  weapon  is  to 
perform  its  mission  and  often  the  specific  technical  characteristics  it  is  t 
possess,  thereby  setting  the  tone  for  the  remainder  of  the  acquisition.  A 
requirement,  for  example,  for  a  manned  strategic  bomber  with  stealth 
features  dictates  the  cost  and  duration  of  development  and  production. 


The  parochialism  endemic  to  the  seiAuces’  structures  for  developing 
weapon  system  requirements  tends  to  narrow  consideration  of  altemati’ 
and  to  favor  the  promotion  of  particular  weapons.  The  convergence  of 
parochial  preferences  and  of  the  demand  for  high  performance  in 
justifying  weapons  creates  incentives  for  developing  requirements  that 
embody  a  preferred  solution  against  which  alternatives  can  hardly 
compete.  The  result  is  not  necessarily  that  the  best  solution  to  a  valid  nc. 
is  selected  but  that  the  preferred  solution  is  successfully  justified. 

The  orgmizations  responsible  for  developing  requirements  for  new 
weapons  generally  represent  individual  branches  within  the  services  tha 
analyze  their  own  mission  area  deficiencies  and  recommend  solutions 
from  within  their  particular  branches.  In  addition,  defense  contractors 
routinely  market  ideas  for  new  weapons  to  the  cognizant  service  brand 
Consequently,  when  an  organization  such  as  the  Anny  Aviation  Center 
identifies  deficiencies  based  on  its  analysis  of  the  threat,  it  proposes 
solutions  in  terms  of  Army  helicopters.  Similairly.  the  responsible  Air 
Force  command  identifies  deficiencies  and  recommends  solutions  in 
terms  of  fixed-wing  aircraft.  This  organizational  alignment  largely  expla 
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why  in  1971,  we  reported  that  the  Air  Force  had  not  included  Army  attack 
helicopters  as  candidates  for  the  close  air  support  mission  (the  A- 10  was 
eventually  selected)'  and  why,  in  the  1980s,  the  Anny  did  not  consider  Ai;'’ 
Force  fighters  as  alternatives  to  developing  the  Comanche  light  helicopter. 

These  service  organizations,  over  time,  have  institutionalized  cin  advocacy 
for  the  weapons  under  their  purview  and  helped  perpetuate  the  funneling 
of  successor  weapons  into  the  acquisition  process.  Under  this  system  for 
developing  requirements,  programs  are  justified  on  the  basis  of  their 
unique  characteristics  under  the  protection  of  closely  guarded  missions, 
making  it  difficult  to  broadly  assess  which  alternative  best  serves  the 
general  defense.  Such  narrow  reviews  of  missions  and  requirements, 
together  with  each  service’s  unwillingness  to  compromise  on  design  or 
performance  goals  for  weapon  systems,  contribute  to  the  services’  large 
investment  in  service-unique  weapons  that  perform  similar  missions. 
Ughter/attack-type  aircraft,  summarized  in  figure  3.3,  are  illustrative:  the 
Navy’s  F-14  and  the  Air  Force’s  F-15  are  both  air  superiority  fighters;  the 
Navy’s  F/A-18  and  the  Air  Force’s  F-IG  are  both  multi-role  fighters;  and  the 
Army’s  attack  helicopters,  the  Air  Force’s  A-10,  and  the  Marine  Corps’ 
AV-8Bs  and  Cobra  helicopters  all  provide  close  air  support  for  ground 
forces.  For  most  of  these  aircraft,  there  is  also  a  program  either  to 
modernize  the  aircraft  or  to  develop  a  replacement. 


’Acquisition  of  Major  Weapon  Systems  (t3-U>j058,  Mar.  18,  Wl\). 
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1 ,600  Attack  Helicopters 
Army 


Air  Force 


Unique  servu.  *  requirements  hawe  led  to  the  acquisition  of  numerous  types  of  aircraft  with 
similar  missions.  (Figures  are  approximate.) 
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Qgrams  Must  Be 
'ersold  to  Survive 


The  Army  attempted  to  reduce  the  parochial  influence  of  its  individual 
requirements  centers  by  having  an  integrating  arms  center  conduct 
combined  mission  area  analyses.  However,  in  a  1986  review,  we  found  that 
reviewers  at  the  new  center  had  the  same  parochial  point  of  view  as  the 
requirements  developers  in  the  individual  centers.^  The  Acting  Secretary  of 
the  Navy,  responding,  in  part,  to  the  problem  of  parochialism,  announced  a 
reorg2inization  of  the  Navy’s  headquarters  staff  in  July  1992.  Tlie  primary 
objective  of  the  reorganization  was  to  better  integrate  the  Navy’s  three 
Wcufare  communities — surface,  air,  and  submcirine — to  end  intraservice 
rivalries  and  to  pursue  integrated  solutions  to  operational  needs. 


Most  programs  are  initiated  as  solutions  to  perceived  deficiencies  in  the 
ability  to  carry  out  missions  against  a  projected  threat  force.  These 
solutions  generally  translate  into  time-critical  needs  for  substantial 
improvements  in  technical  perfonriEince.  Given  the  tendency  to  overstate 
the  capability  of  future  threats,  program  sponsors  are  encouraged  to  write 
demanding  performance  requirements  that  further  distinguish  their 
weapon  as  the  preferred  solution.  Tliese  demands  lead  to  “goldplating,” 
whereby  design  features  are  added,  even  though  their  cost  far  exceeds 
their  real  value.  A  complementary  weakness  in  drawing  up  requirements  is 
that  the  authors  are  not  necessarily  experts  in  technology.  Consequently, 
they  may  lack  the  expertise  to  trade  off  the  increased  cost  and  technical 
risk  associated  with  increased  performance  requirements,  making  it  easier 
to  overstate  needs.  Wlrile  it  may  be  easy  to  criticize  the  authors  of  such 
requirements,  it  must  be  kept  in  mind  that  this  approach  works:  it  is  a 
successful,  if  not  essential,  way  to  win  program  support  from  higher  level 
participants.  By  their  actions,  the  higher  level  participants,  including  the 
Congress,  reinforce  the  formulation  of  demanding  performance  and 
schedule  requirements. 

To  win  support,  programs  not  only  have  to  offer  high  performance  and 
ambitious  schedules,  but  they  must  also  bo  affordable  and  cost-effective. 
These  additional  demands  can  promote  undue  performance,  schedule,  and 
cost  optimism.  For  example,  dod  regulations  require  that  analyses 
assessing  tiie  cost  and  operational  effectiveness  of  various  alternatives 
(including  the  incumbent  weapon)  for  nieeting  a  mission  need  be  prepared 
and  considered  at  major  program  milestone  reviews.  Given  that  the  now 
system  is  already  favored  by  the  service  organization  responsible  for  the 
requirements,  the  cost-effectiveness  analysis  can  further  encourage  the 


’^Weapon  Systems:  Observations  on  Army's  Efforts  to  Improve  Its  Requirements  Process 
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use  of  optimistic  assumptions  of  performance  and  cost.  Contractors 
contribute  to  the  optimism  by  bidding  low  in  their  desire  to  participate  in 
the  program  and  by  proposing  designs  that  promise  exceptional  or  unique 
performance  capabilities. 

In  our  experience,  cost-effectiveness  analyses  rarely  show  an  alternative 
system  to  be  the  preferred  solution.  Even  when  an  alternative  system 
offers  a  potentially  more  cost-effective  solution,  program  sponsors  are  not 
likely  to  be  dissuaded  from  their  preferred  choice.  For  example,  in  our 
recent  review  of  the  Milstar  communications  satellite  program,  we  found 
that  a  1991  dod  study  had  identified  an  alternative  for  satisfying  satellite 
communication  requirements  that  had  a  life-cycle  cost  estimate  that  was 
several  billion  dollars  less  than  that  of  the  recently  restructured  Milstar.  In 
response  to  our  report,  dod  said  that  it  preferred  the  restructured  Milstar, 
citing  larger  cost  uncertainties  and  higher  development  risks  with  the 
alternative,  even  though  its  own  1991  study  had  concluded  that  the  risks 
were  the  same.^ 

Under  the  prevailing  culture,  attributes  such  as  higlt  reliability,  ease  of 
maintenance,  aind  practicality  are  desirable  features  but  are  generally  not 
enough  to  justify  a  program.  The  Army’s  Comanche  helicopter  program  is 
a  case  in  point.  This  program  was  initiated  in  the  early  1980s  by  senior 
Army  management  as  a  family  of  lightweight,  multipurpose  helicopters 
whose  justification  centered  on  practicality  rather  than  the  threat.  The 
program  was  expected  to  inexpensively  replace  a  fleet  of  Vietnam-era 
helicopters  with  new  helicopters  that  would  be  up  to  50  percent  cheaper 
to  operate  m\d  support.  Within  these  economical  confines,  the  new 
helicopters  were  to  offer  as  good  a  technical  performance  as  was  possible. 
Subsequent  to  this  genesis,  however,  specific  requirements  were 
developed  tlirough  the  process  described  above,  and  the  program  emergec 
as  it  is  today — a  threat-based  program  to  yield  the  next-generation, 
high-performance  helicopter  at  a  cost  significantly  higher  than  that  of  the 
Apache,  the  Army’s  most  advanced  and  costly  helicopter. 


ncen  ives  for 
Continuing  Programs 
irow  More  Powerful 
)ver  Time 


As  a  program  proceeds  through  development,  the  disposition  for  sponsors 
to  present  program  information  optimistically  and  to  protect  the  program 
against  disruption  intensifies.  This  behavior  is  necessary  to  overcome  the 
numerous  challenges  a  program  faces  as  it  commands  increasing  funds 
and  faces  potential  criticism.  At  the  same  time,  program  support  grows 


^Military  Satellite  Comniunk’ation.s;  Milstar  Program  Issues  and  Cost-Saving  Opportunities 
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because  more  acquisition  participants  have  become  active  spoi\sors  and 
because  the  time  and  money  invested  have  built  a  compelling  argument  for 
continuing  the  program.  Together,  these  factors  complement  the  initial 
efforts  to  push  the  program  and  begin  to  pull  it  through  the  acquisition 
process.  They  enable  the  program  to  develop  “a  life  of  its  own”  and  to 
become  its  own  objective.  Thus,  even  when  the  very  underpinnings  of  a 
program  are  badly  shaken,  very  strong  arguments  are  made  by  participants 
at  all  levels  to  continue  the  program  as  plaimed.  This  is  particularly  true 
for  programs  that  have  entered  the  engineering  and  manufacturing 
development  phase  (formerly  full-scale  development),  by  which  time  it  is 
generally  conceded  that  ihe  programs  are  committed  to  production. 
Ironically,  if  a  weapon  has  serious  problems,  these  problems  are  most 
likely  to  surface  during  this  phase — when  the  program  has  become 
virtually  unstoppable. 

The  basic  motives  of  prog'-am  sponsors  are  further  galvamzed  by  the 
adversarial  nature  of  repetitive  budget  and  program  reviews.  Collectively, 
multiple  reviews  have  been  a  m.qjor  source  of  instability.  A  program’s 
budget  request  must  survive  the  approval  of  the  service,  the  Office  of  the 
Secretary  of  Defense,  the  Office  of  the  President,  and  at  least  four 
Committees  of  the  Congress.  Each  review  represents  a  threat  to  the 
program  in  the  form  of  potential  budget  cuts  and  program  redirections. 
Because  budget  requests  actually  function  as  vehicles  for  programs  to 
obtain  needed  income,  the  review  process  makes  the  palatability  of 
program  information  more  important  than  its  objectivity.  Thus,  budget 
reviews  can  encourage  further  optimism,  especially  during  the  more  costly 
program  phases. 

Major  programs  must  also  survive  Defense  Acquisition  Board  or 
service-level  reviews  before  they  can  proceed  to  the  next  acquisition 
phase.  These  reviews  can  be  direct  challenges  to  the  program’s  existence 
and  can  demand  proof  or  assurances  that  the  program  (1)  will  meet  its 
performance,  schedule,  and  cost  objectives;  (2)  is  still  a  cost-effective 
solution  to  a  valid  need;  and  (3)  is  stiU  affordable.  At  program  reviews, 
information  outside  the  program  manager’s  control — such  as  independent 
test  evaluations  and  cost  estimates — can  challenge  the  veilidity  of  the 
information  presented  by  the  program  manager.  Even  if  a  program 
survives  a  major  review  intact  and  is  approved  for  the  next  acquisition 
phase,  the  decision  must  then  be  ratified  by  the  Congress  in  terms  of 
funding  and  progr^lm  approvals. 
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When  faced  with  challenges,  sponsors  must  staunchly  defend  their 
program  to  protect  it  and  ensure  its  continuance.  This  discourages  the 
promulgation  of  realistic  information  that  discloses  problems  because  aj 
sign  of  weakness  could  cause  the  Office  of  the  Secretary  of  Defense,  the 
Congress,  or  even  the  service  to  take  funds  from  the  program.  Similarly, 
critical  information  developed  by  oversight  organizations,  such  as  our 
office,  the  Cost  Analysis  Improvement  Group,  the  Director  for  Defense 
Operational  Test  and  Evaluation,  the  Inspector  General,  or  by  the  press, 
will  likely  be  downplayed  and  rebutted  b>  the  sponsors.  Again,  the 
sponsors’  reactions  are  rational,  because  they  realize  that  bad  news  can 
the  scales  of  power  against  the  program,  leading  to  funding  cuts  and 
possible  program  termination.  The  critics’  tendency  to  seize  upon  negati 
information  whenever  it  becomes  available  is  equally  rational,  as  their 
opportunities  to  influence  a  program  can  be  few. 

As  a  program  survives  the  challenges  of  reviews  at  the  service,  Secretarj 
of  Defense,  and  congressional  levels,  many  of  these  participants  become 
program  sponsors.  For  example,  if  a  sendee  acquisition  executive  chairs 
mfyor  review  that  eventually  approves  a  program,  that  executive  will  lik' 
sponsor  and  defend  the  program  to  the  Secretary  of  Defense  and 
Congress.  At  the  congressional  level,  the  voter  constituency  that  would 
benefit  from  the  inco  '  ae  local  contractors  would  receive  tlirough  the 
program  imparts  a  real  pressure  on  Members  to  support  the  program. 
Consequently,  sponsors  can  broaden  the  support  for  a  program  by 
maximizing  the  number  of  states  represented  in  the  contractor  base.  A 
dangerous  situation  from  an  overview  standpoint  occurs  when  the 
principal  decisionmakers  become  program  supporters  before  they  condi 
their  formal  reviews.  Given  the  enormous  advantage  this  situation  gives 
the  program,  a  key  objective  of  program  sponsors  is  to  win  over  the  higl 
levels  of  participants  before  these  higher  levels  formally  review  the 
program.  Consequently,  creating  this  situation  is  a  key  goal  of  lobby 
efforts. 

The  Navy’s  Airborne  Self-Protection  Januner  program  exemplifies  how  j 
weapon  acquisition  can  gather  momentum  as  it  proceeds  through  the 
process  and  how  it  can  become  difficult  to  stop  even  when  serious 
deficiencies  are  known  (see  fig,  3.4). 
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The  testing  criterion  for  this  system  was  adjusted,  enabling  continued  production  despite  known 
deficiencies. 


In  1989,  DOD  approved  initial  production  of  the  Airborne  Self-Protection 
Jammer,  which  is  designed  to  provide  protection  for  F-14  and  F-18  aircraft, 
despite  its  marginal  performance  during  operational  testing,  dod  cited  the 
urgent  need  and  the  absence  of  alternatives  as  the  reasons  for  approval.  In 
1990,  the  DOD  Inspector  General  proposed,  and  we  recommended,  that 
deliveries  from  the  first  lot  be  stretched  out  to  allow  for  complete 
operational  testing  without  a  break  in  production,  dod  did  not  agree  to 
take  this  action,  dod  did,  however,  establish  reliability  and  performance 
criteria  that  had  to  be  achieved  before  the  second  production  lot  would  be 
approved.  When  tests  began  to  show  iliat  system  faiilures  attributable  to 
software  errors  would  cause  the  jammer  to  fall  well  short  of  the  reliability 
critenon,  the  Navy  redefined  tiie  ciiierioii  to  exclude  such  failures.  The 
jammer  met  the  reduced  criterion.  The  Defense  Acquisition  Board, 
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cognizant  that  software  failures  had  been  excluded,  nevertheless  approv 
the  second  lot  following  a  June  1991  program  review.  The  Board’s 
chairman,  the  Under  Secretary  of  Defense  for  Acquisition,  cited  the  neec 
to  avoid  a  production  break  as  a  rationale  for  the  approval.  In 
December  1992,  after  the  jammer  failed  independent  operational  testing, 
the  program  was  terminated.  At  that  time,  136  of  the  approximately  600 
planned  units  were  under  contract. 


Cultural  Differences 
in  Successful 
Programs 


Programs  that  are  successful  in  terms  of  meeting  performance 
expectations  while  remaining  relatively  free  of  cost  and  schedule 
problems  have  operated  within  a  different  set  of  incentives  tlian  most 
other  programs.  Over  the  years,  we  have  reviewed  several  programs  that 
are  considered  successful,  including  the  Fleet  Ballistic  Missile,  F-16 
fighter,  and  Multiple  Launch  Rocket  System  programs.  Because  these 
programs  were  set  up  either  as  special  projects  or  to  achieve  less  risky 
performance  goals,  they  were  spared  the  oversell  and  consequent 
perfoiTOance  bias  that  chaiacterize  most  programs.  Tliese  progr<uns  alsc 
enjoyed  the  continued  funding  support  of  dod  and  the  Congress. 
Consequently,  they  were  less  influenced  by  the  pressiues  that  stem  from 
the  fear  of  technical  failure  and  the  constant  need  to  market  the  progran 
to  secure  funds.  Rather,  these  programs  proceeded  as  executable 
programs  that  did  not  have  to  rely  on  optimism  and  protectionism  to 
survive. 

Successful  programs  have  tended  to  pursue  reasonable  performance 
objectives  and  avoid  the  cascading  effects  of  design  instability  and  the 
equation  of  program  success  with  technical  success.  Freed  from  normal 
acquisition  pressures,  these  programs  were  better  able  to  trade  off 
performance  requii  ements  witli  other  needs  such  as  cost  and 
producibility;  they  have  existed  in  an  environment  where  realism  is 
encouraged.  For  example,  while  the  Fleet  Ballistic  Missile  program  was 
up  in  the  1950s  as  a  concentrated  effort  to  achieve  technical 
breakthroughs  to  field  an  urgently  needed  weapon,  in  the  following 
35  years  it  has  fielded  new  weapons  through  evolutionary  technical 
advances.  Similarly,  the  F-16  was  conceived  as  a  low-cost  alternative  to 
the  F-15  and  did  not  have  as  demanding  perfonnance  requirements  to 
meet.  Consequently,  its  design  was  optimized  for  light  weight  and  low  c( 
and  relied  on  design  features  and  components  that  posed  lower 
performance  and  production  risks.  Furthei.more,  its  design  was  proven  ( 
in  a  competitive  fly-off,  and  changes  were  minimized  by  the  multinationj 
consortium  that  was  to  produce  it. 
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An  unsettling  footnote  to  successful  acquisition  programs  is  the  fact  that, 
in  general,  they  benefit  from  a  set  of  circumstances  not  routinely  available 
to  most  programs.  Unlike  the  Fleet  Ballistic  Missile  program,  most 
programs  cannot  be  set  up  as  special  projects  that  operate  within  a  unique 
set  of  rules.  Nor  have  most  programs  eryoyed  the  flexible  performance 
requirements  instrumental  to  the  F-16’s  success.  Even  the  relative  success 
of  the  Air-Launched  Cruise  Missile  program  can  be  largely  attributed  to  its 
top  priority  and  the  ample  funding  that  enabled  full  competition 
throughout  pilot  production  as  well  as  tlie  dual-sourcing  of  mqjor 
components.  The  challenge  in  extracting  lessons  learned  from  these 
successful  programs  is  in  determining  whether  such  lessons  can  be  used 
to  change  the  culture  in  which  most  programs  must  operate. 
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The  acquisition  culture  offers  an  explanation  for  why  problems  with 
weapon  acquisitions  persist  despite  numerous  attempts  at  reform. 
Reforms,  in  essence,  have  championed  sound  management  practices,  such 
as  realistic  estimating,  thorough  testing,  and  accurate  reporting.  These 
practices,  while  having  improved  acquisition  management,  have  not  been 
widely  adopted  because  they  run  against  the  grain  of,  or  are  at  least 
indifferent  to,  the  very  basic  and  strongly  reinforced  incentives  to  field 
weapons.  In  conti’ast,  practices  not  normally  viewed  as  good  management 
techniques — such  as  concurrency  and  unrealistic  estimates — comprise  an 
effective  stratagem  for  fielding  a  weapon  because  they  reduce  the  risk  tiiat 
the  program  will  be  interrupted  or  called  into  question.  By  default,  many 
improvements  reformers  have  sought  have  become  the  domain  of 
“outsiders”  such  as  regulators,  overseers,  and  critics.  Thus,  not  only  are 
reforms  often  at  cross-purposes  with  the  basic  drives  of  the  acquisition 
process,  their  eventual  position  as  external  controls  can  reinforce  the 
optimism,  parochialism,  and  protectionism  endemic  to  the  process. 


iforms  Have  Battled 
2  Same  Issues  for 
:ars 


It  has  almost  become  a  cliche  to  state  that  the  numerous  reforms  initiated 
over  the  years  have  not  had  the  desired  effect  and  that  today  we  face  the 
same  array  of  acquisition  problems.  The  success  of  “reforms” — defined  in 
the  larger  sense  of  laws,  dod  regulations,  outside  pcuieis,  and 
recommendations  from  independent  agencies — has  been  limited  not 
because  these  reforms  embodied  bad  ideas  or  focused  on  the  wrong 
issues.  To  the  contrary,  reforms  have  generally  been  aimed  at  correcting 
the  most  well  recognized  acquisition  problems,  including  making  cost 
estimates  more  realistic,  reducing  the  time  and  cost  of  acquiring  weapon 
systems,  reducing  dupl’r'*don,  enhancing  program  stability,  improving  the 
quality  of  the  acquisition  work  force,  and  putting  better  information  in  the 
hands  of  decisionmakers  when  they  need  it. 


The  historical  frustration  of  reform  can  be  seen  in  the  action-reaction 
manner  in  which  reforms  have  attempted  to  bridle  the  same  set  of 
problems  over  the  years.  For  example,  two  premier  reforms  arose  in 
response  to  the  problems  experienced  on  the  C-5A  transport  acquisition. 
These  were  dod’s  major  acquisition  regulations  and  Office  of  Management 
and  Budget  Circular  A-109,  which  together  embraced  the  “fly  before  buy” 
concept  and  implemented  the  milestone  decision  approach  to  acquisitions, 
The  C-5A  program  experienced  significant  cost  and  schedule  growth  and 
serious  performance  problems  that  took  years  to  correct.  These  problems 
were  attributed  to  the  over-specification  and  inflexibility  of  performance 
requirements,  the  underestimation  of  technical  risks,  selection  of  the 
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lowest  but  not  most  cost-effective  contractor  proposal,  excessive 
concurrency,  and  an  overreliance  on  fixed-price  contracts  to  manage  a 
sophisticated  development  effort.  Ironically,  the  strategy  used  to  acquire 
the  C-5A — referred  to  as  “Total  Package  Procurement” — was  itself  an 
attempt  to  reduce  cost  overruns,  instill  greater  competition,  assign  the 
contractor  total  responsibility  for  the  design,  and  prevent  contractors  from 
“buying  into”  programs  by  submitting  artificially  low  bids. 

Additional  major  acquisition  reforms  have  sought  to  reduce  cost  and 
schedule  overruns,  to  increase  program  stability,  to  emphasize  realistic 
testing,  and  to  improve  the  efficiency  of  the  acquisition  process.  These 
reforms — which  stretch  back  to  the  McNamara  initiatives  of  the  early 
1960s — have  more  recently  included  the  1981  Defense  Acquisition 
Improvement  Program,  the  1983  Grace  Commission  reports  on  dod,  the 
1986  Packard  Commission  report,  the  1986  Goldwater-Nichols  Act,  and  the 
1989  Defense  Management  Review. 
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Figure  4.1:  Key  Acquisition  Studies  and  Reform  Initiatives _ 

1 960  1 965  1 970  1 975  1 980  1 985  1 990  1 995 

A  McNamara  Initiatives  (1961) 

A  Fitzhugh  Comrriission  (1970) 

A  Commission  on  Government  Procurement  (1972) 

A  Office  of  Management  and  Budget  Circular  A-IOO  (1976) 

A  Defense  Science  Board  Acquisition  Cycle  Study  (1978) 

A  Defense  Resource  Management  Study  (1979) 

A  Defense  Acquisition  Improvement  Program  (1981) 

A  Grace  Commission  (1983) 

A  Packard  Commission  (1986) 

A  Goldwaler-Nichols  Defense  Reorganization  Act  (1986) 
A  Defense  Management  Review  (1989) 


Inability  to  Change 
the  Culture  Has 
Thwarted  Reform 


Despite  the  more  recent  initiatives,  in  the  late  1980s  early  1990s, 
familial'  acquisition  issues  arose,  such  as  the  performance  and 
concurrency  of  the  B-IB  and  B-2  programs  and  the  belated  discovery  by 
DOD  of  significant  cost  overruns  on  the  A-12  aircriift,  v^hich  was  being 
developed  under  a  fixed-price  contract. 


Proposals  to  correct  problems  have  run  the  gamut  of  adding  controls, 
increasing  management  layers,  streamlining,  and  decentralizing.  Wliile 
refoims  have  had  some  success  in  correcting  problems  and  emplacing  a 
system  of  checks  and  balances,  we  have  found  that  after  initiU 
enthusiasm,  support  for  a  reform  wanes.  We  believe  that  acquisition 
refonns  have  had  limited  effectiveness  because  they  have  not  changed  tl 
basic  incentives  or  pressures  that  driven  the  behaxnor  of  the  part,icipant.«!  i 
the  process.  Refonns  have  also  suffered  because  of  acquisition  executivt 
limited  ability  to  effect  cultural  change. 
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Reforms  have  often  sought  coercive,  procedural,  and  organizational 
solutions  to  make  things  happen  without  necessarily  affecting  why  they 
did  not  happen  in  the  first  place.  For  example,  there  have  been 
recommendations  aimed  at  improving  the  realism  of  cost  estimates,  but 
these  are  hard  to  implement  when  the  acquisition  process  itself  does  not 
reward  realism.  The  confrontation  of  reforms  with,  or  their  indifference  to, 
the  acquisition  culture  can  be  illustrated  by  examming  a  few  familiar 
reform  measures:  streamlming,  “fly  before  buy,”  and  baselining. 

Streamlining:  Several  initiatives  have  sought  to  improve  program 
efficiency  and  realism  by  streamlining  and  decentralizing  the  management 
of  acquisition  programs,  The  1981  Acquisition  Improvement  Program 
called  for  the  delegation  of  many  program  decisions  from  the  Secretary  of 
Defense  level  to  the  service  level,  with  the  intention  of  reducing 
“micromanagement.”  Similarly,  the  1986  Packard  Conunission 
recommended  a  three-tiered  organization2il  structure  for  acquisitions 
within  each  service  that  would  reduce  bureaucratic  layers  while  clarifying 
responsibility  and  accountability.  This  structure  was  intended  to  reduce 
the  tendency  of  program  managers  to  be  program  advocates  and  iheivby 
increase  the  flow  of  realistic  information  to  senior  executives. 

These  reform  measures  have  not  ensured  that  more  realistic  program 
information  is  brought  fonvard  in  a  more  timely  manner.  For  example, 
critical  information  about  the  status  of  the  A-12  aircraft  program  v/as 
withheld  by  the  Navy  at  the  time  of  dod’s  1990  mjyor  aircraft  review.  The 
underlying  cultural  pressures  that  cause  such  information  to  be  optimistic 
in  the  first  place  have  not  changed.  The  fact  remains  that  optimistic 
information  helps  a  program  proceed,  while  negative  information  can  hurt 
a  program.  Moreover,  a  primary  source  of  program  information  is  program 
sponsors,  who  are  conditioned  to  be  proponents  of  the  program. 
Streamlining  or  delegating  decisions  may  make  the  process  more  efficient, 
but  it  will  not  necessarily  lead  to  different  outcomes  if  the  same  set  of 
incentives  guides  decisions.  In  fact,  biases  could  become  stronger  if 
decisions  involve  fewer  individuals  whose  livelihoods  are  more 
exclusively  tied  to  the  success  of  weapon  programs. 

“Fly  Before  Buy”:  This  practice  generally  refers  to  building  and 
operationally  testing  prototypes  of  a  weapon  to  ensure  that  the  weapon 
will  work  as  promised  before  r  iiyor  production  commitn'ients  are  made, 
besides  minimizing  problen  •  ui  production,  this  practice  is  intended  to 
provide  decisioumpkers  critical  information  on  .system  Pvaiormanco  and 
risk  so  that  more  informed  decisions  can  be  made.  “Fly  before  buy”  has 
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been  supported  by  public  law,  dod  regulations,  the  Packard  Commissior 
and  the  Defense  Management  Review.  We  have  been  a  strong  supporter 
conducting  sound  operational  testing  early  enough  in  the  acquisition 
process  to  support  production  decisions  and  have  issued  numerous 
reports  on  the  issue.'  Such  testing  has  also  been  institutionalized  in  the 
form  of  operational  test  and  evaluation  agencies  in  each  of  the  services 
and  a  directorate  in  the  Office  of  the  Secretary  of  Defense.  Yet 
concurrency  in  programs — whereby  such  testing  is  conducted  after 
production  has  started — is  a  persistent  acquisition  issue,  as  evidenced  b 
dramatic  test  results  on  the  C-5A,  Sergeant  York,  B-IB,  the  Advanced 
Medium-Range  Air-to-Air  Missile,  and  the  Airborne  Self-Protection  Janui 
programs  after  production  had  begun. 

The  “fly  before  -buy”  approach  to  testing  is  not  resisted  because  it  is 
unsound  or  because  program  sponsors  have  an  affinity  for  increasing 
technical  risks.  Rather,  sponsors’  resistance  to  testing  is  a  logical  reactit 
to  the  additional  time  and  up-front  cost  required  and  to  the  reality  that 
testing  can  jeopeirdize  programs.  This  is  particularly  true  of  operational 
testing  because  it  occurs  outside  of  the  program  manager’s  control, 
exposes  the  weapon  to  harsh  operating  conditions,  and  po  ses  the 
threatening  question  of  whether  the  weapon  can  reliably  execute  its 
mission  t.s  advertised.  Moreover,  although  testing  is  intended  to  be  a 
constructive  tool,  it  tends  to  become  a  report  card  that  surfaces  problen 
at  a  time  when  little  can  be  done  about  them  without  great  effort..  In  fad 
is  not  unusual  for  these  problems  to  be  discoveries  of  latent  design  flaw 
that  should  have  been  caught  earlier  in  the  process.  Tests  represent  key 
opportunities  for  independent  organizations  to  obtain  critical  program 
information  and  can  become  a  lightning  rod  for  debating  a  program’s 
merits.  Thus,  the  rational  use  of  test  results  to  challenge  a  program  evol 
program  sponsors’  rational  aversion  to  testing. 


The  bottom  line  is  that  one  of  the  best  ways  program  sponsors  can 
iixsulate  their  program  from  the  perils  of  the  acquisition  process  is  by 
adopting  a  strategy  of  concurrency,  not  “fly-before-buy.  ”  As  they  see  it,  1 
threat  of  program  disruption  is  a  strong  incentive  for  getting  the 
production  line  started  before  data  from  the  final  phases  of  testing  and 
eva'uation  aite  available.  Problems  can  be  smoothed  out  as  production 
proceeds,  they  reason.  Convet  cly,  if  production  start-up  is  delayed  unti 
deficiencies  are  corrected,  the  program  may  become  stii,inatized,  invitir 
critics  and  budget  cutters  to  conver, 


rri _ , 


‘^b^  further  iiifornialion,  see  Weapons  Testing:  DPI)  Needs  U)  Plan  and  Conduet  Morn  Timely 
Operational  Teste  and  EvaluaUon  (GAO/NSiAU'90-107,  May  17, 1990), 
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poses  risks  that  run  counter  to  sound  management,  it  appeals  to  the 
stronger  motives  of  gaining  commitment  to  a  program  before  negative 
information  can  become  available.  It  is  the  surest  way  of  fielding  the 
weapon  without  interruption. 

Baselining:  This  is  a  technique  aimed  at  enhancing  program  stability, 
whereby  a  program  office  “contracts”  with  top  management  to  develop  a 
system  that  meets  basic  performaitce,  cost,  and  schedule  requirements  in 
exchange  for  stable  funding  and  minimal  interference.  This  approach  was 
embraced  by  the  Air  Force’s  1983  Affordable  Acquisition  Approach  study 
and  the  Packeu-d  Commission  and  is  related  to  the  milestone  authorization 
of  “Defense  Enterprise  Programs”  called  for  in  1986  legislation.  In  theory, 
baselining  makes  sense,  but  it  has  not  met  vrith  great  success  in  practice 
for  two  reasons.  First,  the  guarantee  of  stable  funding  and  noninterference 
requires  that  all  acquisition  participants  commit  themselves  to  multiyear 
funding.  This  is  a  particularly  difficult  commitment  for  the  Congress 
because  it  requires  that  the  Congress  yield  some  of  the  power  of  the 
annual  authorization  and  appropriation  process.  While  the  Congress  has 
been  willing  to  provide  multiyear  funding  for  mature  progreims  in 
production,  it  has  been  reluctant  to  do  so  for  weapons  in  development. 

Second,  the  existence  of  baselining  agreements  or  even  stable  funding 
does  not  ensure  that  estimates  of  performance,  cost,  and  schedule  will  be 
more  realistic  or  attainable.  In  recommending  baselining  as  a  good  way  to 
promote  stability,  the  Packaid  Commission  cited  the  B-IB  as  an  example 
of  a  baselined  system.  Despite  eryoying  stable  funding  and  congres.sional 
support,  the  B4B  has  since  experienced  significant  performance  shortfalls 
and  now  requires  large  sums  of  additional  funding  to  coiTcct  these  and 
other  problems,  From  the  standpoint  of  the  acquisition  culture,  however, 
the  B-IB  was  successful  at  a  very  basic  level:  it  proceeded  through 
development  and  fielding  with  little  interruption.  Similarly,  the  F-1 17A 
program  experienced  many  coiitmon  acquisition  problems  despite  stable 
funding,  continuous  congressional  support,  and  streamlined  management. 
In  January  1991,  we  reported  that  the  F-117A — highly  concurrent  to 
achieve  early  fielding — had  experienced  schedule  delays,  cost  incr(?ases, 
substantial  modifications,  and  several  years  of  substandard  readiness  rates 
(B-238891,  Jan.  11,  1991). 

In  the  final  analysis,  the  objective  of  baselining — stability — has  to  be 
viewed  as  an  essential  ingredient  to  a  siicce.ssful  program.  Conversely, 
destabilizing  a  program  by  altering  performance  requirements,  funding 
a  ailability,  or  schedules  is  almost  certain  to  cause  problems.  However,  in 
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view  of  the  many  pressures  that  characterize  the  acquisition  culture,  stal 
funding  and  support  will  not  prevent  other  acquisition  problems,  such  as 
problems  stemming  from  oversold  performance  requirements  and  liighly 
concurrent  schedules.  In  short,  successful  programs  er\joy  funding 
stability,  but  funding  stability  does  not  ensure  program  success. 


The  short  tenures  tj-pical  of  high-level  UOD  acquisition  executives  make  il 
difficult  for  them  to  change  the  system  of  incentives  because  other 
participants  can  wait  out  reforms  they  oppose.  Even  when  acquisition 
executives  take  firm  stands  to  restrict  specific  programs,  such  as  to  denj 
funding,  they  do  not  have  the  power  to  make  their  decisions  stick.  For 
example,  executives’  decisions  can  be  overturned  by  congressional 
I)aiticipants  who  support  the  program.  Such  has  been  the  case  with  the 
Mjudne  Corps’  AV-8B  Harrier  and  V-22  Osprey  programs.  In  each  case,  thi 
Secretary  of  Defense  excluded  the  programs  from  dod’s  budget  request, 
believing  they  were  not  the  most  cost-effective  solution  to  the  mission 
need.  Congress,  however,  agreed  with  the  Marine  Corps’  professed  need 
for  the  new  systems  and  provided  funding. 

Attempts  by  acquisition  executives  to  obtain  realistic  information  on 
weapon  programs  from  the  military  services  have  also  met  with  limited 
success.  Ill  1990,  we  reported  that  to  protect  programs  from  criticism,  th 
services  were  reluctant  to  provide  the  Office  of  the  Secretary  of  Defense 
current  program  information,  such  as  updated  cost  estimates.'"^  In  fact, 
top-level  acquisition  participants’  dem^mds  for  better  progr^u^l  informatii 
can  intensify  the  protectionism  of  program  sponsors.  For  example,  in 
January  1992,  a  panel  of  the  Nationail  Academy  of  Public  Administration 
reported  that  c  ongressional  demands  for  dod  to  certify  cost,  perfoimanc 
and  reliability  information  on  the  Advanced  Medium  Range  Air-to-Air 
Missile  program,  during  the  1980s  had  instead  yielded  higher  levels  of 
unreliable  information.'*  Similarly,  as  mentioned  earlier,  a  d*  uand  that 
programs  be  affordable  may  be  more  likely  to  result  in  a  les.s  realistic  co: 
estimate  than  in  an  eiffordable  program. 


wtritrnot-  r\K  v}«i 
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DOD  officials  believe  that  the  Defense  Management  Review  initiatives  of 
1989  and  other  actions  the  Office  of  the  Secretary  of  Defense  has  taken  in 
the  past  2  or  3  years  will  address  many  acquisition  problems.  These 
initiatives  include  increased  acquisition  training,  more  independent  cost 
analyses,  and  revisions  to  acquisition  regulations  that  emphasize  a 
technical,  event-based  approach  to  acquisition  management.  This 
approach  includes  a  meyor  effort  to  rigidly  discipline  the  establishment 
and  fulfillment  of  milestone  criteria  before  permitting  a  major  weapon 
program  to  proceed  to  the  next  phase  of  the  acquisition  process  (see 
fig.  1.3). 

Such  an  approach  might  have  been  evident  in  a  1992  Office  of  the 
Secretary  of  Defense  assessment  of  an  acquisition  strategy  proposed  by 
the  Strategic  Defense  Initiative  Organization.  The  critical  assessment  of 
the  strategy  proposed  for  ballistic  missUe  defenses  appears  to  have  been 
instrumental  in  the  decision  to  reduce  risks  in  the  strategy  and  itussile 
deployment  dates. 

Other  initiatives  have  cited  the  need  for  incentives  v/ithin  the  process  Uial 
vdll  motivate  the  individual  services  and  weapon  program  offices  to,  on 
their  own  initiative,  adopt  more  reasonable  program  objectives  and  to  be 
more  candid  about  the  status  of  their  programs.  In  1988,  the  Project  on 
Monitoring  Defense  Reorganization  concluded  the  following: 


In  the  end,  the  success  of  reform  will  depend  on  the  commitment,  the  skills,  the  leadership, 
and  the  dedication  of  the  people  that  make  up  the  defense  establishment.  Very  often  the 
individuals  understand  the  steps  necessary  to  make  the  [Defense]  department  more 
efficient,  but  they  lack  the  incentives  to  ‘do  the  right  thing/ 

Similarly,  the  intent  of  the  Packard  Commission  recommendations  was  to 
nourish  a  new  acquisition  cultm’e  within  dod  so  that  decisions  on 
purchasing  mjuor  weapon  systems  would  be  based  on  realistic  program 
information.  The  extent  of  improvement  in  weapons  acquisition  that  will 
result  from  these  most  recent  initiatives  will  not  be  known  for  several 
years.  It  takes  years  to  develop  a  weapon,  and  because  changes  can  be 
difficult  to  implement  in  ongoing  programs,  it  may  be  5  or  more  years 
before  tangible  improvements  begin  to  be  recognizable  in  new  programs. 
We  believe  the  success  of  these  efforts  will  depend  on  the  cultural  changes 
as  well.  The  delayed  disclosure  of  significant  cost  overruns  on  the  A-12 
illustrates  the  difficulty  of  implementing  change  within  the  existing 

cuituic;. 
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For  quite  some  time,  a  broad  consensus  has  existed  that  weapon 
acquisition  problems  are  seiious  and  their  resolution  is  overdue.  The  fact 
that  such  problems  persist  despite  this  consensus  and  despite  years  of 
reform  underscores  their  resilience.  As  a  practical  matter,  it  must  be 
recognized  that  total  elimination  of  problems  is  not  an  attainable  goal. 
Some  level  of  acquisition  problems  is  to  be  expected,  given  the  risks  and 
complexities  of  weapon  system  programs.  Similarly,  it  is  not  reasonable  to 
expect  that  a  particulai’  reform  or  action  will  be  the  ultimate  solution. 
Rather,  the  goal  of  improving  acquisition  outcomes  is  one  that  requires 
continuous  effort. 

In  keeping  with  this  goal,  we  believe  it  is  an  opportune  time  to  look  at 
acquisition  problems  from  a  different  perspective.  On  the  one  hand,  there 
are  problems  that  are  caused  by  inadequate  training,  lack  of  experience, 
poor  technique,  and  the  like.  It  would  be  proper  to  classify  these  kinds  of 
problems  as  mistakes,  errors  in  judgment,  and  unforeseen  obstacles.  On 
the  other  hand,  there  are  those  problems  that  occur  not  because  they  are 
inadvertent,  but  because  they  are  encouraged.  For  example,  while  some 
problems  in  cost  estiniating  are  due  to  flaws  in  methodology  and  to 
unforeseen  technical  problems,  the  more  pervasive  problem  is  lack  of 
realism.  Such  undue  optimism  does  not  occur  by  chance  or  because 
estimators  lack  know-how,  but  because  it  helps  programs  gain  approval 
and  survive.  It  is  the  latter  problem — symptomatic  of  the  culture — ^that 
acquisition  participants  must  ailso  confront  if  they  truly  want  better 
program  outcomes.  The  opportunities  afforded  by  the  dissolution  of  the 
Soviet  threat  open  the  door  to  making  needed  changes;  declining  defense 
budgets  demaiid  them.  By  themselves,  howevei ,  even  these  compelling 
reasons  may  not  be  enough  to  uproot  an  acquisition  culture  whose  system 
of  incentives  has  become  self-sustaining. 

This  report  does  not  present  the  emswers;  there  are  no  easy  ones. 
However,  in  the  following  pages  we  pose  challenges,  in  the  form  of 
questions,  we  believe  can  help  acquisition  participants  change  the 
incentives — and  the  culture — of  weapons  acquisition.  These  views 
necessarily  involve  judgments  and  are  subject  to  debate.  However,  the 
specifics  of  these  challenges  should  not  overshadow  their  general  intent, 
which  is  to  spotlight  the  acquisition  culture  as  a  proper  focus  of 
prescriptions. 

Acquisition  participants  hold  the  key  to  cultural  change  since  they  largely 
deter  Afunc  ti>6  ulOtiVeS  ^iCl  iuCtJUtiVOs  of  tht?  iiCqUlsiUuii  Thu  llisl 

Step  toward  such  change  is  for  the  acquisition  participar\ts  to  accept  their 
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collective  responsibility  for  the  incentives  that  drive  the  process,  rather 
than  to  place  blame  on  individuals  or  procedure. 


e  Participants 
lling  to  Make 
crifices  to  Achieve 
tter  Outcomes? 


While  much  debate  and  reform  have  centered  on  the  “how”  of  acquisitions 
(procedures  and  control)  and  the  “who”  (organization),  a  cultural  focus 
must  address  the  “why” — the  motivations  and  incentives  that  affect 
behavior.  In  this  vein,  the  key  “problem”  with  the  acquisition  process  may 
be  that  it  does  such  a  good  job  of  meeting  the  diverse  needs  of  its 
pairticipants.  This  seeming  contradiction  is  consistent  with  the 
predominant  outcomes  of  the  acquisition  process.  Over  the  years,  we  have 
observed  that,  while  a  small  number  of  systems  reach  the  field  as 
unqualified  successes  arid  a  small  number  are  canceled,  most  weapons 
reach  the  field  but  cost  more,  take  longer,  and  are  harder  to  produce  iind 
support  than  expected.  Generally,  these  weapons  represent  a  significant 
improvement  over  those  they  replace,  but  it  is  not  uncommon  for  them  to 
have  performance  shortfalls  or  to  require  expensive,  time-consuming 
modifications  to  achieve  the  required  performance.  Even  special  access, 
or  “black”  programs,  although  subject  to  less  public  scrutiny,  have  similar 
outcomes. 


It  is  the  consistency  of  these  outcomes — and  their  imperviousness  to 
reforms,  contract  types,  contractors,  acquisition  strategies,  weapon  types, 
critics,  military  services,  administrations,  at\d  Congresses — that  leads  to  a 
conclusion  that  the  acquisition  process  may  be  producing  what  the 
participants  collectively  v,'ant  or  are  willing  to  settle  for. 

Instead  of  pointing  the  finger  at  individual  participants,  one  must  keep  in 
mind  that  they  do  not  act  irrationally  or  with  bad  intentions.  Rather,  they 
do  what  they  believe  is  right,  given  the  pressures  they  face.  The  difficulty 
lies  in  the  fact  that  there  is  no  consensus  on  what  is  right.  In  the  absence 
of  such  a  consensus,  the  acquisition  process  serves  to  satisfy  the  diverse 
needs  of  its  participants  within  the  umbrella  of  providing  U.S.  forces  with 
the  best  weaponry.  In  so  doing,  the  incentives  of  the  process — both 
positive  and  negative — favor  maximizing  progrjuns.  Parochieilism, 
optimism,  protectionism,  and  information  hoarding  are  pragmatic 
responses  to  these  incentives,  while  cost  growth,  schedule  delays, 
duplication,  and  perfonnance  problems  are  the  logical  consequences. 

Ultimately,  predominant  program  outcomes  are  consistent  with  this  set  of 
incentives.  If  better  program  outcomes  are  desired  and  persistent 
acquisition  problems  are  to  be  alleviated,  then  the  motives  and  incentives 
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that  drive  the  participants  and  the  process  must  be  realigned  to  produce 
such  outcomes.  The  challenge  lies  in  the  fact  tliat  better  outcomes  will 
require  participants  to  give  up  something — ^to  ask  the  acquisition  process 
to  satisfy  fewer  needs.  If  there  is  to  be  less  duplication  among  programs, 
more  achievable  objectives,  and  more  candor  about  results,  then  there  w 
have  to  be  fewer  programs,  smaller  ones,  £md/or  programs  with  less 
ambitious  requirements.  The  question  is  whether  participants  are  willing 
to  make  the  sacrifices  needed  to  achieve  these  outcomes. 


!  Participants 
ling  to  Recognize 
Broader 
Tsequences  of 
ividual  Actions? 


A  powerful  influence  on  the  behavior  of  program  sponsors  and  skeptics  i 
their  perception  of  “what  works.”  Thus,  it  is  the  actions  taken  on 
individual  progrcuns  that  define  incentives  and  motives  more  than  what  b 
prescribed  on  paper.  Within  the  current  process,  compelling 
circumstances  can  make  a  case  for  taking  actions  on  individual  programf 
that  would  otherwise  coiitradict  policy  or  compromise  sound  principles. 
Yet  it  is  these  individual  actions — not  policy — that  communicate  the 
broader  message  of  “what  will  work”  to  others  in  the  process. 

For  example,  a  decision  to  approve  a  program — such  as  the  Airborne 
Self-Protection  Jammer — ^for  production  despite  incomplete  developmen 
may  be  right  for  the  program,  but  it  sends  the  message  to  other  programs 
that  concurrency  and  skirting  test  criteria  are  acceptable.  Similarly,  whei 
the  Secretary  of  Defense  takes  the  difficult  stand  to  disapprove  a  prograr 
a  decision  by  the  Congress  to  restore  it  tells  program  sponsors  that  such 
decisions  can  be  overturned.  The  difficulty  lies  in  the  fact  that  when 
looked  at  individually,  circumstances  can  almost  always  make  a 
compelling  case  in  favor  of  the  program  at  hand.  Thus,  on  a  case-by-case 
basis,  decisions  to  approve  prograros  despite  their  potential  duplication, 
known  problems,  or  their  failure  to  live  up  to  advertised  expectations  cai 
be  rationalized  in  a  number  of  ways:  there  is  an  urgent  threat  to  be  met;  £ 
w£urm  production  base  needs  to  be  m£iintained;  despite  shortfalls,  the  nei 
system  is  more  capable  than  the  one  it  is  replacing;  the  new  system’s 
problems  can  be  fixed  later. 


While  we  do  not  advocate  the  blind  application  of  policy  to  all 
circumstances,  we  do  believe  that  decisions — whether  by  dod  or  the 
Congress — to  approve  progrcuiLS  that  embody  problems  reinforce  those 
problems.  Thus,  if  duplicative  programs  are  approved  despite  an 
acquisition  policy  to  the  contrary,  the  message  to  other  program  sponsor 
and  skeptics  is  that  duplication  is  acceptable.  The  challenge  for  SCQUiSiti 
participants,  particularly  for  those  with  program  approval  authority,  is  tc 
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treat  individual  program  decisions  as  more  than  the  case  at  hand.  They 
must  weigh  the  broader  implications  of  what  message  tliese  decisions 
send  to  other  participants  regarding  what  is  acceptable  or  “what  will 
work.” 


11  Participants 
;ree  to  a  Military 
rategy  Before 
iking  Major 
iquisition 
immitments? 


The  defense  acquisition  decisions  made  over  the  next  few  years  will  be 
especially  critical  because  they  eire  intertwined  with  the  rewriting  of 
national  security  policy  and  military  strategy.  Decisions  on 
next-generation  weapons  will  define  solutions  to  defense  policy  needs, 
dictate  budgets  for  the  remainder  of  the  decade,  and,  in  the  process,  either 
take  advantage  of  or  miss  the  opportunity  to  improve  the  acquisition 
culture.  While  much  debate  will  continue  to  center  on  potentially  dramatic 
changes  in  the  level  of  spending  and  the  number  of  new  weapon  programs 
to  be  funded,  these  changes  will  not  necessarily  lead  to  better  outcomes 
on  those  progreuns  that  survive.  In  fact,  problems  on  individued  weapon 
programs  could  worsen  if  the  intensified  competition  for  funds,  prompted 
by  the  reduced  threat  and  declining  budgets,  leads  to  increased  optimism 
as  a  tactic  necessary  for  program  sui’vival. 

Wlrile  DOD  has  revised  the  military  strategy,  congressional  debate  on  issues 
such  as  roles  and  missions  suggests  that  a  consensus  has  not  been 
reached.  Long-term  dividends  could  result  if  the  Congress  and  the 
administration  refrain  from  making  weapon  system  milestone  decisions 
until  they  agree  on  a  military  strategy,  including  how  smd  where  U.S. 
forces  should  be  prepared  to  fight;  how  the  forces  should  be  structured  to 
accomplish  national  security  objectives;  and  how  to  preserve  the  research, 
technology,  and  industrial  base.  These  should  be  explicit  choices;  they 
should  neither  be  dictated  by  individual  program  decisions,  nor  should 
they  be  the  tenuous  byproduct  of  budget  compronuses.  With  em 
agreed-upon  strategy,  consensus  on  the  roles  and  missions  of  the  services 
could  be  more  easily  reached,  and  weapon  progrmns  could  then  be  the 
result  of  clear  decisions  on  how  to  implement  policy,  rather  than  the  result 
of  incremental  decisions  that  assuage  individual  interests. 


an  Incentives  Be 
hanged  to 

iscourage  Optimism, 
arochialism,  and 
rotectionism? 


The  preceding  challenges,  if  collectively  accepted  by  acquisition 
participants,  could  lay  the  groundwork  for  cultural  change.  The  following 
are  more  specific  suggestions  to  further  institute  such  change.  Because  a 
discussion  of  culture  necessarily  involves  motives,  beliefs,  and 
perceptions,  more  specific  suggestions  can  and  should  evoke  clo.ser 
debate.  Regardless  of  the  specific  actions  participants  take  to  effect 
change,  we  believe  the  objectives  should  be  twofold:  (1)  to  uproot 
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traditioneil  seats  of  parochialism  and  self-interest  and  (2)  to  enforce  a  set 
of  incentives — ^both  positive  and  negative — ^to  motivate  participants  to 
take  actions  that  are  consistent  with  better  program  outcomes. 


’he  Authority  for  To  reduce  the  parochialism  currently  inherent  in  program  justifications, 

)eterrnining  Program  the  determination  of  needed  capabilities  and  the  particular  types  of 

Jeeds  weapons  to  fill  those  needs  should  not  be  left  with  individual  branches  am 

warfare  communities  within  the  services.  The  parochial  and  duplicative 
outcomes  of  the  acquisition  process  are  consistent  with  the  fact  that 
system  requirements  mirror  the  traditions  and  self-preservation  instincts 
of  their  sponsoring  organizations.  Moving  this  authority  higher  up  in  the 
DOD  organization  could  enable  competing  demands,  available  resources, 
and  the  needs  of  theater  commanders  to  be  more  fairly  assessed  before  a 
specific  program  need  is  given  life.  There  is  not  necessarily  a  “right” 
answer  as  to  where  the  authority  for  determining  program  needs  should  b 
revested.  Po.s.sibilitie.s  include  an  enhanced  Joint  Requiremerts  Oversight 
Council,  which  currently  reviews  program  requirements;  the  Defense 
Planning  and  Resources  Board,  which  was  established  to  help  develop 
stronger  links  between  national  policies  and  the  resources  allocated  to 
specific  programs  and  forces;  the  Office  of  the  Secretary  of  Defense;  or 
perhaps  a  high-level  council/board  within  each  service,  following  any 
realignment  of  their  roles  and  missions.  It  is  also  important  that  if  a  debati 
is  to  occur  between  dod  and  the  Congress  over  the  need  for  a  weapon,  it 
should  occur  early  in  the  process  before  the  weapon  gains  too  much 
momentum. 


’he  Need  to  Oversell 
*rograms  on  the  Basis  of 
’op  Performance  and 
Jrgency 


The  dissolution  of  the  Soviet  Union,  together  with  the  current  U.S. 
inventories  of  highly  capable  weapons,  represents  opportunities  to  abate 
the  need  to  oversell  weapon  programs,  the  associated  optimism  in  cost 
and  schedule,  and  the  tendency  to  weaken  acquisition  strategies  in  favor 
of  schedule  acceleration  or  preservation,  dod  has  made  several  proposals 
along  these  lines,  including  limiting  or  temunating  production  plans  for 
several  major  weapon  systems  and  reducing  concurrency  in  new 
programs.  However,  these  changes  will  not  necessarily  produce  better 
program  outcomes  if  overselling  performance  and  urgency  still  “work”  in 
gaining  program  approval.  Defusing  the  need  to  oversell  programs  on  the 
basis  of  performance  and  urgency  may  involve  several  steps.  First, 
lup-ievei  uecisioiuuakers  in  uou  and  the  Congress  should  agree  to  a  poiici 
regarding  “what  will  work”  that  is  consistent  with  desired  outcomes.  For 
example,  if  the  national  security  strategy  suggests  less  need  for 
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revolutionary  performance  gains  on  future  weapons,  then  an  acquisition 
policy  could  emphasize — and  thus  create  incentives  for — ^programs  whose 
technical  advances  are  aimed  at  reducing  duplicative  systems  and  support 
costs. 

Second,  individuals  with  pertinent  technical  expertise  and  knowledge  of 
the  state-of-the  art  of  key  technologies  available,  including  those  in  the 
commercial  sector,  should  be  more  involved  in  defining  the  specific 
performance  required  of  a  weapon  system.  Such  discipline  could  better 
balance  the  reasonableness  of  performance  requirements  against  their 
cost,  risk,  and  value.  According  to  dod  officials,  more  of  tltis  type  of 
analysis  is  being  done.  Infusing  such  expertise  into  requirements  definition 
could  be  facilitated  if  the  ‘.cthority  for  determining  requirements  were 
moved,  as  discussed  earlier 

Third,  and  perhaps  most  important,  decisionmalcers  must  ensure  that  their 
decisions  on  individual  programs  are  consistent  with  their  desire  to 
discourage  tlie  overselling  of  programs. 

Other  pressures  that  influence  the  justification  of  weapon  systems  could 
also  be  relieved  through  up-front  policy  agreements  among  acquisition 
participants.  For  example,  it  may  be  preferable  to  declare  and  pursue  a 
policy  of  replacing  some  mtOor  platforms  on  a  set  schedule  rather  than 
tying  the  need  for  their  replacement  primarily  to  performance 
advancements.  Also,  as  the  defense  industrial  base  recedes  in  response  to 
smaller  budgets,  it  may  make  sense  to  develop  a  list  of  critical  capabilities 
and  facilities — including  those  to  be  provided  in-house,  by  contractors, 
and  by  universities — ^that  need  to  be  preserved  in  the  interest  of  national 
security.  If  an  acquisition  program  is  justifiable  as  an  effective  means  of 
preserving  a  critical  industrial  capability,  this  justification  should  be 
candidly  debated  at  the  outset  to  relieve  the  program  manager  of  the  need 
to  oversell. 


len  to  Define  Programs  Carrying  research  and  development  efforts  further  before  incorporating 

them  into  specific  weapon  progreims  could  reduce  the  tendency  to 
overpromise  expected  results.  With  the  traditional  emphasis  on  high 
performance,  weapon  programs  have,  cut  of  necessity,  relied  on  risky 
technology  development  efforts.  Such  efforts,  when  drawn  into  a  mjyor 
program,  not  only  become  dedicated  to  the  program  but  are  subjected  to 
the  same  requirements  for  precise  cost  and  schedule  estimates,  even 
though  their  immaturity  defies  such  precision.  Moreover,  test  results  from 
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ancial  Realism 


such  efforts,  like  test  results  in  general,  are  used  by  proponents  to  help 
market  the  program  and  by  opponents  to  criticize  it.  The  former  Under 
Secretary  of  the  Army  has  cited  such  premature  program  definition  as  a 
msyor  cause  of  turmoil  in  the  Comanche  Light  Helicopter  program. 

Freeing  research  and  technology  efforts  from  program  association  until 
they  mature  to  a  specified  level,  such  as  the  demonstration  and  validation 
phase,  could  be  one  element  of  ein  overall  strategy  to  ensure  that  the 
nation  nurtures  a  healthy  research  and  technology  base  in  the  face  of 
declining  weapons  production.  Given  sufficient  funding,  the  efforts 
themselves  would  benefit  because  they  would  be  more  able  to  explore  the 
full  range  of  results,  rather  than  being  directed  toward  meeting 
program-specific  goals.  Under  these  circumstances,  testing  could  assume  a 
“no  fault"  nature,  whereby  its  main  and  proper  purpose  would  be  to  gauge 
and  guide  the  progress  of  the  work.  In  this  arena,  test  failures,  problems, 
and  redesigns  would  be  part  of  a  healthy  process,  whereas  the  same 
results  now  represent  potential  disasters  in  mgjor  programs. 

When  the  need  for  a  mgjor  program  is  determined,  the  program  could 
“cherry  pick”  from  mature  reseairch  and  technology  projects,  reducing  the 
subsequent  risks  to  the  program.  In  addition,  the  testing  of  a  m^or 
weapon  could  then  be  more  properly  focused  on  “how  well”  it  i^;rforms, 
rather  than  on  “whether”  it  will  perform.  Many  of  these  ideas  Fxe  already 
embodied  in  dod’s  new  acquisition  strategy,  which  calls  for  greater 
demonstration  of  advanced  technologies,  to  prove  their  feasibility  and 
producibility,  before  incorporating  them  into  new  or  ongoing  acquisition 
programs.  The  success  of  this  strategy  will  depend  on  the  cooperation  of 
all  acquisition  participants. 


In  the  private  sector,  realism  is  essential  to  a  responsible  individual’s 
solvency  and  to  a  commercial  enterprise’s  profitability;  undue  optimism 
has  direct  and  dramatic  consequences.  Conversely,  in  weapon 
acquisitions,  optimistic  cost  estimates  are  rewarded  because  they  help 
gain  program  approval,  win  contract  awards,  and  attract  budgetary 
income.  The  consequences  of  cost  growth  are  not  directly  felt  by  an 
individual  program  because  they  are  “accommodated”  through 
stretch-outs  and  quantity  changes  and  by  spreading  the  pain  across  many 
programs. 


To  discourage  unrealistic  cost  estimates,  the  consequences  should  be  tied 
back  to  the  program  at  hand.  Such  incentives  could  work  the  other  v/ay  as 
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well — ^programs  that  are  well-managed  within  estimates  should  be 
undisturbed  and  should  keep  at  least  a  portion  of  any  cost  savings  they 
can  achieve.  One  approach  would  be  to  require  an  up-front  determination 
of  how  much  money  can  be  afforded  for  a  new  program,  bounded  by  a 
range  that  reflects  uncertainty.  That  is,  the  range  of  costs  should  be  wider 
early  in  a  program  when  uncertainty  is  greater  and  narrowed  as  the 
program  matures.  This  having  been  done,  subsequent  cost  increases  would 
have  to  be  absorbed  by  the  program  itself,  either  in  the  form  ^f  reduced 
quantities  or  reduced  capabilities.  The  idea  is  not  to  overemphasize 
discipline  to  the  extent  that  '^alid  unanticipated  needs  cannot  be 
accommodated.  However,  such  accommodatioris  should  be  the  exception; 
they  are  currently  the  rule. 

The  Future  Years  Defense  Program  can  be  used  as  a  tool  to  help  decide 
how  much  money  can  be  afforded  for  an  individual  program  and  to 
confront  the  consequences  of  program  cost  grovvlh.  The  Future  Years 
Defense  Program  could  force  the  arrangement  of  programs  so  they  fall 
within  reasonable  funding  levels.  The  timing  of  major  programs  could  be 
staggered  to  reflect  financial  realities.  Without  some  actions  along  these 
lines,  DOD  could,  in  the  future,  be  faced  with  a  financial  “bow  wave”  of 
next-generation  weapons — a  condition  that  can  bring  out  the  worst  in 
acquisition  mmiagement.  The  funding  plan  could  also  serve  to  discourage 
other  sources  of  program  cost  growth,  including  requirements  increases 
and  program  redirections.  When  any  participant — including  the 
Congress — proposes  an  action  that  will  change  the  funding  profile  or 
timing  of  a  program,  that  participant  should  also  propose  the  trade-offs  in 
the  plan  that  will  make  the  action  fiscally  possible. 

A  final  step  in  tlie  arena  of  fineincial  incentives  could  involve  the  timing  of 
budget  requests.  Currently,  budget  requests  must  often  presume  outcomes 
from  future  program  events.  For  example,  the  budget  request  for  a 
program  scheduled  for  operational  testing  in  November  1993  and  a 
production  decision  in  April  1994  must  be  submitted  to  Congress  in 
January  1993.  To  meet  this  budget  submission  date,  the  program  manager 
must  actually  develop  an  individual  program  budget  request  in  early  1992 
for  review  and  approval  within  lxjd.  Not  only  does  this  sequence  of  events 
force  the  program  manager  to  .spend  a  lot  of  time  defending  the  budget 
request  absent  critical  information,  but  it  also  provides  more  incentive  to 
push  the  program  through  the  events  to  save  the  budget.  To  relieve  this 
pressure,  consideration  should  be  given  to  having  budget  requests  that 
implement  m^or  decisions  follow — not  precede — those  decisions. 
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One  of  the  aspects  of  the  acquisition  culture  that  will  be  most  difficult  to 
change  is  the  fact  that  the  success  of  participants’  careers  is  more 
dependent  on  getting  programs  through  the  process  than  on  achieving 
better  program  outcomes.  The  success  of  cultural  change  will  depend  on 
whether  participants  are  rewarded  for  taking  actions  that  produce  better 
outcomes.  We  believe  the  challenges  discussed  above  can  provide  some  of 
those  rewards.  It  is  also  possible  that  program  managers’  careers  could  be 
aligned  with  better  outcomes  if  their  progression  is  governed  by  the 
quality  of  their  management  and  not  the  survival  of  their  programs. 

However,  the  picture  is  less  clear  for  other  participants.  For  example,  dod 
acquisition  executives  are  critical  to  cultural  change  because  of  their 
control  over  large  numbers  of  programs  and  because  they  are  in  a  position 
to  enforce  a  set  of  incentives  within  dod  that  rewards  better  outcomes. 
However,  tliey  do  not  necessarily  stay  in  their  positions  long  enough  to 
develop  the  needed  long-term  perspective  or  to  effectively  change 
traditional  incentives.  Moreover,  tlreir  decisions  can  be  overruled  through 
the  cooperative  actions  of  other  participants.  Similarly,  it  is  difficult  to 
envision  Members  of  Congress  being  relieved  of  pressures  to  support 
programs  that  benefit  their  constituents,  despite  efforts  to  change  the 
culture. 

At  this  point,  perhaps  the  most  important  step  participants  can  take  is  to 
recognize  the  broader  implications  of  their  individual  actions.  In  July  1992, 
the  Chairman  of  the  Senate  Armed  Services  Committee  called  for  an 
overhaul  of  military  roles  and  missions.  To  make  the  difficult  choices 
necessary  in  such  an  overhaul,  he  suggested  that  the  .standard  should  be 
“what  is  best  for  Americei,’’  not  what  is  best  for  the  individual  services. 
This  standard  should  govern  the  actions  of  weapons  acquisition  process 
particip- ’-^ts  at  aU  levels  as  well. 


We  have,  over  the  yeaurs,  taken  the  position  that  strong  central 
management  on  the  part  of  the  Office  of  the  Secretary  of  Defense  is 
essential  to  acquisition  reform.  Such  management  is  crucial  now,  given  the 
significant  future  consequences  of  the  decisions  being  made.  Once  a 
healthier  culture  is  established,  the  need  for  centralization  may  eventually 
be  lessened.  Although  we  often  disagree  with  dod  on  acquisition  matters, 
we  believe  that  the  top  management  in  the  Office  of  the  Secretary  of 
Defense  and  the  Joint  Chiefs  of  Staff  has  displayed  the  ability  and 
conviction  to  forge  significant  change.  They  have  aiso  done  much  to 
reestablish  central  management  of  weapon  acquisitions.  In  addition  to 
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dod’s  recent  reform  initiatf.'es,  these  characteristics  have  been  manifested 
in  a  number  of  ways,  including 

•  the  strong  collective  leadership  of  the  Secretary  of  Defense  and  ihe 
Chainnan  of  the  Joint  Chiefs  of  Staff  during  Operation  Desert  Storm; 

•  dod’s  proposal  to  trade  off  weapon  programs  in  favor  of  military  jjersonnel 
in  an  effort  to  make  an  orderly  transition  to  a  smaller  force; 

•  more  realistic  funding  projections,  coupled  with  an  announced  rin  a  stand 
not  to  allow  programs  to  proceed  if  they  are  shown  to  be  unaffordable  in 
the  future;  and 

•  renewed  commitment  to  ihe  Packard  Commission’s  reconiinendations  and 
to  improviivg  the  quality  and  professionalism  of  the  acquisition  work  f<.)rce. 

The  Congress  ha  also  taken  constructive  actioris  and  made  proposals  to 
improve  weapon  acquisitions.  For  example,  the  Joint  Chiefs  of  Staff  was 
strengthened  by  the  Goldwater-Nichols  Act,  enabling  it  to  function  more 
effectively  in  situations  such  as  Operation  Desert  Storm.  Both  the  Senate 
and  House  Am\ed  Services  Committees  have  been  forthright  in 
highlighting  tlic  need  for  a  new  militarj^  strategy  in  light  of  the  reduced 
threat  and  have  put  forth  proposals  on  such  a  strategy.  The  Senate  Armed 
Services  Conmiittee  has  pioneered  efforts  to  authorize  funding  for  entire 
acquisition  phases,  so  as  to  reduce  program  instability.  More  recently, 
Members  of  Congress  have  proposed  renewed  emphasis  o:  “fly  before 

buy”  policy  in  weapon  programs  and  a  civilian  acquisition  a.  ^  to 
reduce  the  acquisition  bureaucracy  and  make  program  management  more 
effective. 

Today,  the  ingredients  for  making  lasting  improvements  to  weapons 
acquisii  ions — the  need,  the  opportunity,  and  the  leadership — exist.  To 
convert  these  ingredients  into  constructive  change  will  require  both  the 
Congress  and  the  administration  to  take  joint  ownership  of  repetitive 
acquisition  problems  and  to  take  explicit  steps  to  resolve  them.  The 
actions  already  underway  are  important  to  better  outcomes.  Also 
important,  in  our  view,  is  the  recognition  of  the  cultural  dimension  of 
acquisition  problems  and  the  solutions  it  suggests.  Beyond  directives  and 
controls,  acquisition  partic  i})ants  will  have  to  pull  together  to  make  better 
outcomes  the  more  natu  products  of  a  healthier  acquisition  culture. 
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7  able  I.l  provides  a  brief  overview  of  weapon  system  issues  we  have 
identified  over  the  past  15  years. 


3ie  1.1 :  Weapon  System  Programs  and  Issues 

Army 

he  system  the  best  solution  to  the  mission  need?  -  Terminal  Guidance  Warhead 

-  Armored  System  Modernization  program 

-  Dragon  i!  antitank  weapon 

-  Air  Defense  Antitank  System 

-  Apache  attack  helicopter 

-  Near-term  Scout  helicopter 

-  Viper  antitank  weapon 

ij  the  program  cost  and  schedule  estimates  realistic?  -  Javelin  antitank  system 

-  Terminal  Guidance  Warhead 

-  Comanche  light  helicopter 

-  Air  Defense  Antitank  System 

-  Bradley  Infantry  Fighting  Vehicle 

-  Armored  Systems  Modernization  program 

-  Black  Hawk  helicopter 

-  M-1  Abrams  tank 

-  Stinger  antiaircraft  missile 


in  the  program  be  executed  with  a'^ailable  funds? 

-  Comanche  light  helicopter 

-  Apache  attack  helicopter 

'  Armored  Systems  Modernization  program 

the  program's  acquisition  strategy  reasonable? 

-  Army  Tactical  Command  and  Control  System 

-  Air  Defense  Antitank  System 

-  Comanche  light  helicoptei 

-  Special  operalionr  torces  helicopter 

-  M-1  Block  11  tank 

-  Armored  Systems  Modernization  Program 

-  Sergeant  Vork  Division  Air  Delense  gun 

as  it  been  shown  ttiat  the  system  is  operationally  effective? 

-  Air  Defense  Antitank  System 

-  Bradley  Infantry  Figtiting  Vehicle 
'  Aquila  remotely  piloted  vehicle 

-  Sergeant  York  Division  Air  Defense  gun 

-  Viper  antitank  missile 

-  Copperhead  projectile 

-  Standoff  Target  Acquisition  System 

-  Bigeye  bomb 

;  the  system  suitable  for  production  and  fielding? 

-  Air  Defense  Antitank  System 

-  Apache  attack  helicopter 

-  Aquila  remotely  piloted  vehicle 

-  Patriot  dir  defense  system 

-  Coppcrtieaci  projectile 

-  Stinger  antiaircraft  missile 

-  Black  1  lawk  helicopter 

Page  70 


GAO/NS  I  AD-93- 15  Laating  Change  In  V/eapoiis  Acqui^ili 


Appendix  1 

Examples  of  Weapon  System  Acquisition 
Issues  Reported  by  GAO  From  January  1978 
Through  September  1992 


i/y 

Air  Force 

adar  jammers 

22  Osprey  tiltrotor  aircrafi 

y-8B  Harrier  aircraft 

apidly  Deployable  Surveillance  System 

spendable  Reliable  Acoustic  Sonobuoy 

dvanced  Signal  Processor 

icticai  towed  array  sensor  systems 

“  Milslar  communications  satellite 

-  Radar  jammers 

-  Sensor  Fuzed  Weapon 

-  Advanced  Warning  System 

-  Tactical  Air  Navigation  System 

-  U.S.  Antisatellite  Program 

-  Over-the-Horizon  Backscatter  Radar 

•22  Osprey  tiltrotor  aircraft 
'leigh  Burke  DDG-51  destroyer 

DE-6  fast  combat  support  ship 

Diiing  Airframe  Missile 

Ba  Lance  missile 

A- 18  fighter 

AVSTAR  Global  Positioning  System 

Harrier  airciafl 

ght  Airborne  Multi-Purpose  System  MK  III 

-  Short-Range  Attack  Missile  II 

-  Advanced  Tactical  Fighter 

-  C-17  .'i'rlift  aircraft 

-  F-1 17A  fighter  aircraft 

-  Advanced  Medium  Range  Air-to-Air  Missile 

-  B-1B  and  B-2  strategic  bombers 

-  FF-1 11 A  aircraft 

-  U.S.  Antisatellite  Program 

-  Low  Altitude  Navigation  and  Targeting  Infrared  System 

3awoltSSN-21  submarir^e 

rieigh  Burke  DDG-51  ucjstroyer 

ght  Airborne  Multi-Purpose  System  MK  IN 

-  B-2  strategic  bomber 
■  Peacekeeper  missile 

-  Small  Intercontinental  Ballistic  Miss  ile 

-  U.S.  Antisatellite  Program 

jaw'  -‘f  SSN  21  subfranne 
rieigh  Burke  DDG-51  aestroyer 
oiling  Airframe  Missile 
iK-48  and  MK-50  torpedoes 
hcenix  AIM-54C  missile 

ityiiic;i  cmvvidii 

aptor  Ocean  Mine  System 

-  B-lB  and  B-2  strategic  bombers 

-  C-17  airlift  aircraft 

-  Peacekeeper  Rail  Garrison  missile 

-  Advanced  Cruise  Missile 

-  Advanced  Medium  Range  Air-to-Air  Missile 

-  r-1 17A  fighter  aircraft 

-  ACU-130  gunship 

oiling  Airframe  Missile 
iK-48  and  MK-50  torpedoes 
amahawk  Land-Attack  Cruise  Missile 

V-8B  Harrier  aircraft 
‘14D  fighter  aircraft 

AVSTAR  Global  Positioning  System 
urveillance  Towed  Array  Sensor  System 
-inch  Major  Caliber  Lightweigti^  Gim 

-  Small  intercontinental  Ballistic  Missile 

-  Peacekeeper  missile 

-  B-1B  and  B-2  strategic  bombers 

-  JP'233  airfield  attack  system 

-  Wide  art  a  antiarmor  munitions 

-  Precision  Location  Strike  System 

-  Low-Altitude  Ballistic  Missile  Defense  System 

-22  Osprey  tiltrotor  aircraft 
lecironic  warfare  test  equipment 
halanx  close-in  v/eapon  system 
igh-3peed  Anti-Radiation  Missile 

V-88  Hamer  aircraft 

1  Oriiior>  K  yf  i*-*  i  I 

-  Electronic  /warfare  test  equipment 

-  B-1B  and  B-2  strategic  bombers 

-  Advanced  Cruise  Missile 

-  Advanced  Medium  Range  Air-io-Air  Missile 

-  F-1 17A  fighter  aircraft 

1  HJIl  ItJ  tT  »«i  l_UII 

-inch  Major  Caliber  Lightv/eight  Gun 

*  PeacckGGpcr  missile 

P\)!lowing  an;  discussions  of  several  spiHafic  examples  diat.  illustrate  each 
of  the  six  issues.  Neither  table  I.l  nor  th(!S(!  (examples  are  intended  to  be 
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all-inclusive.  However,  they  are  presented  to  illustrate  how  widespread  the 
acquisition  problems  discussed  in  chapter  2  have  been. 


the  System  the  Best 
)lution  to  the 
ission  Need? 


In  addition  to  the  Sensor  Fused  Weapon  program  cited  in  chapter  2,  other 
specific  examples  of  programs,  or  mission  areas,  in  which  the  need  for  the 
weapon  system  was  in  question  or  consideration  of  alternative  solutions  to 
mission  needs  was  a  key  issue  include  the  following: 


•  In  June  1992,  we  reported  on  uod’s  restructuring  of  the  Milstar 
communications  satellite  program  (gao/nsiad-92-i2i).  In  restructuring  the 
program,  dod  plans  included  adding  a  medium  data  rate  capability  to 
satellites  number  4  and  beyond.  We  noted  that  dod  had  not  demonstrated 
that  this  capability  on  Milstar  would  be  the  most  cost-effective  means  of 
satisfying  tactical  communications  requir'-^menls.  Two  loss  costly 
alternatives  that  meet  requirements  were  identified  in  an  October  1991  dod 
study.  One  alternative  was  characterized  as  having  moderate  development 
risk  (the  same  as  the  restructured  Milstar),  and  the  other  as  ha\diig  high 
risk.  The  life-cycle  costs  for  the  moderate  risk  alternative  was  estimated  to 
be  several  billion  dollars  less  over  a  20-year  period  than  the  restructured 
Milstar.  For  the  high  risk  alternative,  the  20-year  life-cycle  costs  were 
estimated  to  be  several  billion  dollars  less  than  even  the  moderate  risk 
alternative. 

•  In  an  April  1992  report,  we  discussed  the  acquisition  of  Army  and  Air 
Force  weapons  for  the  close  support  mission  of  attacking  targets  in  close 
proximity  to  friendly  forces  (gao/NS1AD-uz-iho).  The  identification  of 
weapons  to  perform  this  mission  is  controlled  by  the  military  services, 
which  g(merate  their  own  mission  needs  assessments.  These  assessments 
have  not  questioned  what  and  how  much  is  i  leeded;  nor  have  they  fully 
coi\sidei  ed  the  expected  contributions  of  other  close  support  v/eapon 
systems.  The  consideration  of  alternative  weapons  to  satisfy  a  mission 
need  is  generally  limited  to  a  single  type  of  weapon.  For  example,  the  Air 
Force  s  1990  assessment  of  how  to  replace  the  A-10  fixed-wing  aircraft 
considered  only  other  fixed  wing  aircraft — it  excluded  Army  artillery 
systems  and  attack  helicopters,  dod  considers  each  type  of  close  support 
weapon  syst.em  to  be  “complementary"  to  the  others  and,  therefore,  all  are 
necessary.  The  resultant  ac:quivSition  decisions  provide  no  assurance  that 
DO!)  has  selected  the  most  cost-effective  solutions  to  meet  close  support 
mission  needs. 

•  In  November  1991,  we  report.ed  on  i)C)i/s  plans  to  develop  a  folluw-on 
system  to  the  Defense  Sujiport  Program,  a  strategic  surveiiiance  and 
warning  satellite  system  ((jao/nsiaimj^:39).  We  cited  five  studies  that 
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indicated  t^e  new  follow-on  system  non  was  planning  to  develop  may  not 
be  the  most  cost-effective  solution  to  the  mission  requirements. 

Indications  were  that  an  upgraded  system  would  pro\1de  most  all  of  the 
capability  needed  at  a  significantly  lower  cost.  The  new  system,  as  initially 
deployed,  would  also  not  be  capable  of  fully  meeting  mission 
requirements,  but  would  be  designed  to  accommodate  future 
enhancements  to  meet  requirements.  However,  the  incremental  costs 
necessaiy  to  bring  initial  capabilities  up  to  the  full  requirements  had  not 
been  estimated  by  the  Air  Force.  Study  estimates  indicated  that  when  all 
costs  were  considered,  the  new  warning  system  would  cost  about 
$3  billion  more  than  an  upgraded  system. 

•  Several  of  our  reports  over  the  past  10  yeais,  including  two  classified 
reports,  have  identified  an  unwarranted  proliferation  of  electronic  warfare 
systems  to  protect  aircraft  against  potential  threat  weapons.  '  For  example, 
the  Air  F’orce  and  the  Navy  have  12  different  radar  jammers  in  use  or  being 
acquired  at  a  cost  exceeding  $9  billion  to  protect  tactical  fighter  and  attack 
aircraft  against  a  common  threat.  None  are  used  by  both  services.  This 
proliferation  has  occurred  despite  dod’s  commitment  and  long  standing 
congressional  interest  aimed  at  promoting  commonality. 

•  In  a  classified  March  1983  report,  we  recommended  that  the  Air  Force 
fully  reassess  the  need  to  continue  development  and  procurement  of  the 
Ovci-the-Horizon  Backscatter  radar  system.  The  program  included  the 
development  of  four  radar  systems  to  provide  a  long-range  tactical 
warning  capability  to  help  counter  a  threat  of  a  Soviet  bomber  atl^ack  on 
North  America.  We  pointed  out  that  both  the  Air  P'orce  and  the  Navy  were 
planning  to  develop  better  warning  systems  for  the  1990s  and  that  until 
the 36  systems  were  available,  existing  airborne  warning  and  control 
system  aircraft  could  be  used  to  strengthen  tactical  waniing  capabilities. 
The  Air  Force  continued  the  program,  which  fell  behind  schedule  and  was 
terminated  in  January  1991,  with  two  of  the  four  radar  systems  having 
been  delivered  by  the  contractor.  The  program  acquisition  cost  was  about 
$1.45  billion  (in  escalated  dollars). 

•  In  a  classified  1981  report,  we  reported  that  the  Army's  Viper  portable 
antitank  missile  wuuld  have  only  limited  effectiveness  against  the  threat  it 
would  face.  We  concluded  that  it  should  not  be  produced  and 
recommended  that  other  alternatives  be  examined.  Tlie  Army  decided  to 
proceed  with  the  Viper  and  awarded  a  production  contract  in 
December  1981.  However,  that  same  month  the  Congress  directed  the 
Army  and  the  Marine  Coi^^s  to  lest  other  available  alternative s.  These  tests 
revealed  that  none  of  the  alternatives,  including  the  Viper,  could 


'Navy/Air  .Still  I )(*vf‘)(;))ing  Soparalc,  Costly  Radar  Warning  Kccriv^TS  ( ( r At  >/NSIAI  )-K7- j  (iV. 
July  1,  1987);  Multiple  l)(!V(*l()i)m<*iit.  (jf  Co.st.ly  'rhr(!al  Siinu.Uitors  ('tiAO/NSlAlJ-88-98.  F(»h.  1.  1988); 
Klf>(^t.rontr  Warfare:  Radar  JainnuT  ProlifcTation  (VuiliniiPs  (OA(VNSlAI)-92-HJ^,  I'V'h.  'AH,  H)92). 
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effectively  penetrate  the  frontal  armor  of  modem  tanks.  The  Army 
dropped  its  requirement  for  a  light,  portable  antiarmor  weapon,  selected  a 
Swedish  system  for  further  testing,  and  teiminated  the  Viper  production 
contract. 


In  addition  to  our  September  1991  report  on  cost  and  schedule  overruns 
on  missile  systems,  discussed  in  chapter  2,  other  reports  on  weapon 
system  programs  in  which  we  cited  cost  and  schedule  issues  as  important 
problems  or  concerns  include  the  following: 


In  September  1992,  we  reported  that  the  cost  estimates  for  the  DDG-51 
shipbuilding  contracts  showed  that  the  first  three  ships  of  this  new 
destroyer  program  will  total  $1.1  billion,  double  the  oiiginal  contract 
award  cost  estimates.  As  a  result  of  major  contract  modifications  and 
contract  cost-sharing  provisions,  the  Navy  will  pay  for  most  of  the 
increases.  Officials  of  Bath  Iron  Works,  the  prime  contractor  for  the  lead 
ship,  described  to  us  in  1985  how  their  company  had  bid  “aggressively”  for 
the  DDG-51  lead  ship  contract  The  company  believed  its  survival 
depended  on  getting  Na\7  contracts  for  ships  in  the  DDG-51  program, 
considering  the  lack  of  commercial  shipbuilding  in  the  United  States. 

In  September  1992,  we  reported  that  the  acquisition  cost  estimate  for  the 
Ai*my*s  Javelin  antitank  system  had  increased  over  $1  billion  since 
June  1989,  a33-percont  increase  ((;Ao/NSiAi)-n2-;3ao).  The  increase  was 
attributed  primarily  to  (1)  stretching  out  the  production  schedule  from  6  to 
10  years  because  of  funding  constraints  and  (2)  technical  problems  that 
led  to  additional  delays.  We  also  reported  that  the  cost  of  this 
58,000-system  prograin  could  increase  further  because  the  production  cost 
estimate  for  a  key  component  appeared  opUmistic. 

In  several  classified  repoits,  the  latest  issued  in  July  1992,  we  cited 
problems  on  the  Tri-Hervice  Standoff  Attack  Missile  program,  including  a 
high  degree  of  concurrency,  technical  complexity,  and  higlily  optimistic 
plans  and  estimates,  ^’he  cumulative  (Tfect  of  these  problems  has  been 
program  cost  increases  in  exccuss  of  50  percent,  more  a  4-year  growth 
in  the  development  program,  and  a  m£\jor  restructuring  of  the  fixed-price 
development  contract.  Development  of  this  highly  classifi(?d  missile  began 
in  1986. 

In  April  1992,  we  repoitcd  that  th(‘  (M7  transj)ort  aircraft  development 
program,  which  included  six  prcKluction  airc*raft,  was  continuing  to 
expenence  significant  cost  and  schedule  problems  ((iAo/NsiAiMj^-^onuK). 
Cusis  iin.:uitcu  by  ilu*  cuiu-rciCi.o  r  had  exceeds*  d  Uie  contract  ceiling  price 
with  over  1('  perccait  of  tlie  work  remaining.  The  government’s  estimate  of 
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the  cost  at  completion  was  also  continuing  to  increase  and  stood  at 
$7.45  billion,  $813  million  over  the  contract  ceiling.  Additional  increases 
have  since  occurred.  One  of  the  primary  causes  of  the  cost  and  schedule 
increases  was  the  inability  of  the  contractor  to  produce  the  aircraft  at  the 
levels  of  efficiency  used  in  developing  the  contract  estimates. 

•  In  a  classified  December  1991  report,  we  discussed  the  President’s 
termination  of  the  strategic  Short-Range  Attack  Missile  II  program  in 
September  1991.  We  reported  that  at  the  time  of  termination,  the  program 
schedule  had  slipped  about  4  years;  the  estimated  unit  cost  had  nearly 
doubled,  from  about  $0.8  million  to  $1.4  million;  and  the  quantity  of 
missiles  to  be  bought  had  decreased  from  1,633  to  700.  These  changes 
were  partially  attributable  to  very  serious  development  problems  that 
affected  the  capability  of  the  missile  to  accomplish  its  mission. 

•  The  Navy’s  V-22  Osprey  Vertical  Lift  Aircraft  program,  has  experienced 
both  development  delays  and  unit  procurement  cost  growth.  In  an 
October  1990  report,  we  attributed  this  growth  to  c.h^lnges  in  the  program’s 
structure,  continuing  development  problems  that  could  seriously  affect  the 
aircraft’s  performance,  cancellation  of  production  options,  the  Army’s 
'.vithdrawal  from  the  effort,  and  the  Air  Force’s  reduced  buy 
(gao/n.siad-91-4.'’)).  As  of  April  1992,  the  full-scale  development  effort  was 
about  3  years  behind  schedule.  The  estimated  unit  procurement  cost  had 
increased  to  $32.5  million  (fiscal  year  1992  escalated  dollars)  and  was 
expected  to  rise  above  $40  million  by  time  the  aircraft  was  delivered. 

These  high  costs  prompted  the  Navy  in  August  1992  to  look  into 
alternatives  to  the  V-22  and  to  ask  the  V-22  contractors  to  design  a  lower 
cost  version  of  the  aircraft. 

•  In  May  1984,  we  issued  a  report  on  the  effectiveness  of  dod’s 
cost-estimating  processes  (gao/nsiad-h-i-to).  The  report  was  based  on  a 
review  of  the  processes  used  on  seven  weapon  system  programs — the 
Apache  helicopter,  the  Bradley  Fighting  Vehicle  System,  the  Hellfire 
Missile,  the  Light  Airborne  Multipurpose  Helicopter,  Landing  Ship  Dock 
(LSD)  -41,  the  T-46A  Trainer,  and  the  B-IB  bomber.  We  found  weaknesses 
in  the  cost-estimating  guidance  provided  program  offices  and  a  need  for 
more  realism  in  the  assumptions  and  methodologies  used  in  preparing 
cost  estimates.  W(‘  found  the  use  of  overly  optimistic  assumptions  in  areas 
such  as  construction  schedules  and  allowances  for  uncertainties.  For 
example,  in  preparing  the  Helifire  iroduction  cost  estimate,  the  Army  had 
not  performed  an  engineering  risk  analysis,  desjiite  knovm  significant 
shortcomings  in  some  ma,jor  Hellfire  components.  We  also  found  that  boo 
did  not  always  thoroughly  consider  the  lecommendations  of  its 
iudcpciKlont  cost  groups  iu  prcpciriug  progrsjtu  cost  Gstiru2.t6S.  This 
happened,  for  examjile,  on  the  Bradley  program  when  an  independent 
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estimate  cited  significant  cost  risks  with  the  vehicle’s  fire  control  system. 
No  change  was  made  to  the  program  estimate  to  reflect  this  risk.  The 
following  year  the  contractor’s  cost  estimate  for  the  fire  control  system 
rose  $336  million. 

•  In  February  1982,  we  reported  that  the  AV-8B  Harrier  program  cost 
estimate  did  not  include  over  $1  billion  in  program  costs — over 
$700  million  for  treuner  aircraft  and  over  $300  million  for  an  on-board  gun 
system  (masad-82-i9). 


Specific  weapon  systems  on  which  we  have  raised  questions  regarding  the 
affordability  of  the  program  include  the  following: 

•  In  April  1990,  we  reported  that  executing  the  Navy’s  29-ship  Seawolf 
submarine  program  within  projected  funding  hinged  on  four  optimistic 
assumptions:  fl)  a  sustained  shipbuilding  and  conversion  budget  grovnh 
of  3  percent  above  inflation,  (2)  a  reduction  in  submarine  construction 
time  by  more  than  1  year,  (3)  congressional  authorization  for  an  average  of 
three  SSN-21s  per  year,  and  (4)  no  cost  overnms  in  the  program 
(GAO/NSiAD-90  if)3).  A  subsequent  dod  major  warship  review  recognized  these 
inherent  affordability  problems,  and  as  a  result,  the  Secretary  of  Defense, 
in  1991,  reduced  the  SSN-21  program  to  12  ships  to  be  procured  over  a 
9-year  period. 

•  In  April  1987,  we  raised  several  concerns  over  the  affordability  of  the  Air 
Force’s  plan  to  modernize  and  expand  its  tactical  fighter  force  from  36  to 
40  wings  (gao/nsiad-87-121).  The  plan  was  premised  on  the  assumptions  that 
defense  spending’s  real  growth  would  continue  and  that  the  share  of  the 
Air  Force  budget  being  allocated  to  tactical  Air  Force  programs  would 
increase.  We  pointed  out  that  neither  of  these  premises — which  allowed 
for  increased  projections  of  available  funds — was  likely.  Moreover,  we 
noted  that  future  tactical  aircraft  programs  would  be  competing  with  high 
priority  progreuns  such  as  the  C-17  cargo  plane  and  the  Advanced 
Technology  Bomber  (B-2)  programs.  The  Air  Force  acknowledged  these 
affordability  concerns  and  delayed  expansion  of  the  tactical  fighter  wings. 

•  The  Army’s  Comanche  light  helicopter  program  quantities  have  been 
significantly  reduced  over  the  years  as  a  result  of  growing  costs  and 
limited  funds.  Originally,  the  Army  had  planned  to  buy  5,023  of  these 
helicopters,  at  a  peak  production  rate  of  480  per  year.  In  March  1987,  we 
reported  that  quantities  had  dropped  to  4,500  because  of  program  cost 
increases  (GAO/NSlAD-rv-iiTt'S),  By  June  1988,  quantities  had  dropped  to 
2,096  as  a  rfsnlt  of  additional  cost  growth,  with  apeak  production  rate  of 
216  per  year.  We  reported  in  December  1988  that  even  this  reduced 
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program  did  not  appear  affordable  because  available  funding  was 
sufficient  for  a  production  rate  of  only  157  per  year  (gao/nsiad-89-72).  In 
1990,  the  Army  reduced  program  quantities  again  to  1,292  helicopters,  with 
a  peak  yearly  production  rate  of  120.  In  May  1992,  we  reported  that 
because  of  the  rising  unit  cost  of  the  Comanche,  dwindling  defense 
resources,  the  diminished  threat,  and  planned  upgrades  to  other 
helicopters,  the  Army’s  requirements  for  this  aurcraft  should  be  reassessed 
(gao/nsla.d-92-204).  The  Secretary  of  Defense  had  earlier  announced  that  dod 
will  concentrate  on  making  small  numbers  cf  prototypes  of  the  Comanche 
to  develop  the  technology  and  capability.  It  will  not  automatically  move 
into  full-scale  production. 


Is  the  Program’s 
Acquisition  Strategy 
Reasonable? 


Examples  of  weapons  programs  or  reports  in  which  either  the  overlap 
between  development  and  production  or  the  timing  and  quality  of 
operational  testing  of  the  weapon  system  were  concerns  include  the 
following: 

In  August  1992,  we  reported  on  dod’s  incremental  development,  testing, 
and  fielding  strategy  for  the  Army’s  Tactical  Command  2uid  Control  System 
(gao/nsiad-92-151).  Under  this  strategy,  dod  planned  to  conxmit  to 
production  about  $1.8  billion  in  equipment  for  the  system  before  ensuring 
that  all  five  of  the  system’s  segments  would  work  together  as  intended. 

DOD  defended  its  acquisition  strategy  on  the  grounds  that  it  provided  the 
soldier  with  a  usable  segment  of  the  system  as  it  became  available.  The 
system,  however,  had  been  encountering  continuous  development 
problems  and  test  failures.  For  the  system  to  perform  its  mission,  all  five 
control  segments  had  to  be  effectively  integrated.  The  Army  had  already 
wasted  $155  million  in  premature  equipment  acquisitions  for  the  system, 
and  the  urgency  to  field  it  vastly  diminished  with  the  dissolution  of  the 
Soviet  threat. 

In  March  1992,  we  issued  a  report  on  dod’s  1991  decision  to  continue 
production  of  the  Navy’s  Airborne  Self-Protection  Jammer  despite 
reliability  probleins  because  of  a  desire  to  avoid  a  break  in  production 
(gao/nsl\d-92-103).  In  justifying  this  production  decision,  the  Navy 
discounted  certain  software-induced  failures,  thus  circumventing  certain 
DOD  testing  standards  and  failing  to  recognize  the  adverse  impacts  of 
software  problems  experienced  with  other  similar  electronic  warfare 
systems.  For  example,  in  July  1990,  we  reported  that,  because  of  software 
problems,  the  Air  Force  had  to  place  improved  ALQ-135  Jammers 
produced  for  F-15  aircraft  in  storage  rather  than  deliver  them  to  tactical 
units  (gao/nsiad-90'168).  Again,  in  December  1987,  we  reported  that 
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production  began  on  a  classified  Air  Force  warning  receiver  before  testing 
began.  The  Air  Force  later  found  that  the  performance  or  the  receiver  was 
worse  tlian  that  of  the  receiver  it  was  intended  to  replace.  The  Air  Force 
continued  to  use  the  older  warning  receiver  and  to  produce  the  newer 
receiver,  only  to  place  it  in  bonded  storage  pending  redesign  to  solve  the 
problem. 

•  In  a  May  1989  report  on  Navy  testing,  we  disclosed  that,  as  an  alternative 
to  the  more  realistic  field  testing  of  weapon  systems  in  support  of  a 
decision  to  enter  full-scale  development  or  low-rate  production,  non  was 
encouraging  operational  assessments  based  on  modeling,  simulation,  or 
paper  analyses  (gao/nsiad  89-98).  We  reported  that  such  assessments,  while 
providing  useful  information  in  assessing  the  development  status  of  a 
weapon,  do  not  provide  the  realistic  conditions  generally  needed  to  ensure 
accurate  assessments  for  decision-making.  In  response  to  our  report, 
which  was  based  on  a  review  of  19  Navy  systems,  the  Congress  enacted  an 
amendment  that  excludes  such  operational  assessments  from  the  statutory 
definition  of  “operational  test  and  evaluation.” 

•  The  Navy  had  to  place  Phoenix  AIM-54C  missiles  in  storage  for  extended 
periods  of  time  pending  design  ehanges  to  solve  problems  with  a  critical 
missile  component  discovered  during  tests  but  not  resolved  before  missile 
production  began  (gao/nsiai)-88-i04).  At  the  time  of  our  March  1988  report 
on  the  quality  of  the  Phoenix,  the  Navy  had  already  accepted  deliveiy  of 
over  500  nonfunctional  missiles  valued  at  about  $472  million.  A  Navy 
official  at  the  weapons  center  overseeing  development  of  the  missile 
component  told  us  the  center  had  failed  to  adequately  monitor  the 
program  because  it  “wanted  the  hardware  out  the  door.” 

•  In  August  1987,  we  reported  tha^  non  had  approved  initial  low-rate 
production  of  the  Advanced  Medium  Range  Air-to-Air  Missile  despite 
knowing  that  the  design  was  likely  to  change  because  many  of  the  more 
demanding  tests  had  not  been  completed  ((iA()/NSiAi)-87-ir)8).  Performance 
and  reliability  problems  were  subsequently  identified  in  testing,  but, 
despite  these  problems,  dod  approved  production  of  arlditional  missiles  in 
1988.  These  missiles  required  numerous  modifications  to  the  hardware  and 
software  to  correct  the  reliability  and  performance  problems  found  in  the 
test  program.  These  modifications  placed  an  additional  burden  on  the 
production  program,  which  was  already  struggling  and  well  behind 
schedule, 

•  In  December  1986,  we  issued  a  report  summarizing  problems  reported 
since  1970  on  the  (juality  of  dod  operational  test  and  evaluation 
((;AO/NSiAD-y7.57).  Wc  reported  that  the  usefulness  of  operational  test  and 
evaluation  in  estimating  a  weapon  system’s  performance  had  been  limited 
because  of  long-standing  problems  in  test  planning,  test  conduct,  and  the 
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reporting  of  test  and  evaiuation  results.  For  example,  we  had  reported  on 
32  historical  cases  in  which  testing  was  unrealistic  because  it  did  not 
adequately  replicate  the  operational  environment;  25  cases  in  which  test 
objectives,  criteria,  and  plans  were  incomplete,  unclear,  and/or  absent 
before  testing  began;  27  cases  in  which  test  resources  were  limited  or  not 
available;  and  22  cases  in  which  test  reports  did  not  always  contain  the 
most  current,  complete,  or  accurate  data  on  the  performance  of  m^or 
systems  before  production. 


as  It  Been  Shown 
lat  the  System  Can 
Brform  Effectively? 


Examples  of  programs  on  which  the  effectiveness  of  the  weapon  in 
accomplishing  its  niission  requirements  was  a  significant  issue  we 
reported  on  include  the  following: 

•  In  February  1992,  we  reported  on  the  technological  challenges  that  must 
be  overcome  under  the  Strategic  Defense  Initiative  to  proceed  with  the 
Global  Protection  Against  Limited  Strikes,  a  defense  system  that  uses  both 
ground-  and  spaced-based  interceptors  (gao/imtI':c-92-18).  Such  a  system 
pushes  the  cutting  edge  of  technology.  System  developers  will  have  to  rely 
on  some  technologies  that  are  not  yet  proven  and  learn  how  to  integrate 
them  into  a  reliable  system.  In  a  March  1992  report,  we  described  how 
estimates  of  the  effectiveness  of  Brilliant  Pebbles,  the  proposed 
space-based  weapon  system,  were  based  on  relatively  immature  computer 
simulations  that  used  many  unproven  assumptions  about  key  operational 
and  performance  characteristics  (gao/nsiau-92-9i).  Considering  the 
uncertainties  involved  in  a  system  of  this  complexity,  we  reported  there 
was  a  significant  risk  that  the  level  of  protection  promised  would  not  be 
achieved. 

•  In  August  1990,  we  reported  on  the  results  of  our  joint  review  with  the 
German  Federal  Court  Audit  of  the  Rolling  Airframe  Missile  program 
((;ao/nsiad-90-208).  The  United  States  and  Germany  had  agreed  to  jointly 
develop  and  produce  the  missile  to  provide  close-in  defense  against 
antiship  cruise  missiles.  Our  analysis  showed  that  the  Rolling  Airframe 
Missile’s  capabilities  would  be  stressed  or  surjiassed  by  most  of  the 
antiship  missiles  currently  deployed  or  planned.  The  number  and  the 
capability  of  antiship  missiles  in  various  regions  of  the  world  had 
increased  significantly  since  development  of  the  missile  had  begun.  These 
missiles  had  advanced  performance  characteristics  for  which  the 
countering  technology  had  yet  to  be  developed,  pod  and  the  German 
Ministry  of  Defen.se  flid  not  concur  with  the  majority  of  our  findings  and 
recommendations.  Congre.ss,  however,  terminated  the  program,  citing  the 
concern.s  expressed  in  our  report. 
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•  In  April  1990,  we  reported  that  problems  with  embedded  computer 
resources,  particularly  software,  were  occurring  more  and  more  frequently 
and  causing  many  weapon  system  cost  overruns,  schedule  delays,  and 
performance  shortfalls  (gao/imtec-90-34).  A  recent  example  is  the  Navy’s 
F-14D  aircraft.  In  April  1992,  we  reported  that  the  F-14D  software 
contained  serious  defects  that  prevented  the  aircraft  from  meeting  its 
operational  requirements  (gao/imtec-92-2i).  Defects  in  the  aircraft’s 
embedded  software  have  caused,  for  exeimple,  cockpit  displays  to  go 
blank  and  erroneous  data  to  be  supplied  to  the  mission  computer.  In 
addition,  some  originally  planned  software  needed  to  support  additional 
avionics  and  weapons  capability  was  deferred  due  to  development 
problems.  At  the  time  of  our  report,  the  Navy  had  already  procured 

43  F-14Ds,  all  of  which  will  be  delivered  to  the  Navy  without  software 
required  to  meet  mission  objectives.  Another  exan\ple  is  the  C-17  aircraft. 
In  a  May  1992  report,  we  discussed  how  the  Air  Force  had  assumed  that 
software  was  a  low-risk  effort  and  had  done  little  to  manage  its 
development  or  to  oversee  the  contractor’s  performance  (gao/imtec-92-48). 
Problems  occurred,  and  software  development  has  fallen  behind  schedule. 
Software  testing  shortcuts  were  being  taken  to  limit  the  impact  on  the 
schedule.  Such  shortcuts,  however,  raise  the  risks  that  software  problems 
will  go  undetected  uruil  the  aircraft’s  flight  test  phase. 

•  In  October  1987,  we  reported  on  operational  testing  of  the  Army’s  Aquila 
remotely  piloted  vehicle  (gao/nsiad-8819).  The  test  results  showed  that  the 
Aquila  would  not  perform  its  mission  effectively.  In  105  test  flights,  the 
system  successfully  performed  all  of  the  mission  requirements  only 

7  times,  it  experienced  mqjor  difficulties  in  consistently  achieving  a 
successful  launch,  in  detecting  targets,  and  in  reaching  acceptable 
reliability  and  maintainability  levels.  Major  concenis  also  surfaced  with 
the  survivability  of  the  aircraft  and  with  its  being  able  to  be  used 
proficiently  by  intended  operators.  The  Congress  did  not  fund  the  Aquila 
in  fiscal  year  1988,  and  the  Army  never  pursued  it  into  production. 

•  Live-fire  testing  in  1985  of  the  Army’s  Bradley  F'ighting  Vehicle  showed 
that  the  vehicle  was  highly  vulnerable  to  all  antiarmor  weapons 
(GAO/NSiAD-874i0.  The  test  results  and  Army  vulnerability  models  further 
showed  that  ammunition  stored  inside  the  vehicle,  when  hit,  was  likely  to 
be  the  major  source  of  crew  casualties  and  vehicle  losses.  Enhancements 
were  made  to  the  vehicle,  but  these  only  protected  against  the  less 
effective  enemy  antiarmor  weapons.  For  the  Bradley  to  achieve  an 
acceptable  level  of  survivability,  the  Army  planned  to  rely  on  the  vehicle’s 
mobility  and  firepower  and  on  tactics  to  avoid  overmatching  enemy 
weapon  systems,  whenever  possible.  If  survivability  testing  had  been 
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performed  earlier  in  the  acquisition  process,  design  changes  inight  have 
been  possible,  thereby  improving  crew  and  vehicle  survivability. 

•  In  April  1979,  we  reported  that  recent  battle  experience  and  various  Navy 
studies  had  shown  that  many,  if  not  most,  of  the  surface  ships  that  would 
be  in  the  U.S.  fleet  during  1975-90  had  not  been  adequately  designed  to 
absorb  and  control  damage  from  so-called  “cheap  kills”  O’Sad-79-43). 

Weapon  system  performance  and  cost  were  being  emphasized  in  ship 
design  at  the  expense  of  survivability.  For  example,  vital  electronic 
equipment  was  being  located  in  highly  exposed  areas.  This  allowed  for 
better  performance  but  made  the  equipment  easily  susceptible  to  damage 
from  such  things  as  small  metal  fragments  from  near  misses  of  missiles  or 
rockets.  Measures  to  enhance  ship  survivability  were  estimated  to  cost 
several  billion  dollars.  A  limited  number  of  improvements  were 
implemented. 

•  The  Navy’s  8-inch  Mcyor  Caliber  Lightweight  Gun  for  surface  combatant 
ships  was  confronted  with  significant  performance  problems.  In  our 
August  1978  classified  report  on  the  gun,  we  noted  that  internal  dod 
studies  had  raised  que.stions  about  the  advisability  of  procuring  it.  The  gun 
was  to  be  deployed  on  Spruance  class  destroyers,  which  probably  would 
not  be  available  for  amplubious  assault  operations  or  shore 
bombardment — two  of  the  gun’s  primary  missions — because  the  Spruance 
class  destroyers  would  likely  be  committed  to  higher  priority  roles.  Also, 
the  range  of  hostile  missiles  was  much  greater  than  the  gun’s  range,  which 
could  force  the  destroyers  to  remain  at  distances  beyond  the  effective 
range  of  the  gun.  Despite  these  and  other  concerns,  the  Secretary  of 
Defense  approved  the  gun  for  limited  production.  The  Congress,  however, 
refused  to  support  acquisition  of  the  gun,  and  the  program  was  terminated. 


[s  the  System  Suitable 
for  Production  and 
Fielding? 


Specific  examples  of  programs  for  which  the  ability  of  the  contractor  to 
produce  it  or  t  he  ability  of  the  sei^lces  to  support  the  system  was  an  issue 
include  the  following: 


•  In  May  1992,  we  reported  on  concerns  with  the  Air  Force  s  Advanced 
Cruise  Missile,  including  problems  with  quality  that  have  plagued  the 
program  for  several  years  and  contributed  to  lengthy  delivery  delays 
(GAO/NSiAD-92-154).  Although  production  quality  was  improving,  the  Strategic 
Air  Command  was  continuing  to  identify  manufacturing  defects  in  missiles 
during  receipt  inspections.  While  some  of  these  were  minor,  3  of 
26  missiles  inspected  by  the  Command  between  November  1991  and 
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schedule  called  for  delivery  of  all  of  these  high  priority  missiles  by  the  end 
of  fiscal  year  1990.  As  of  March  1992,  only  40  had  been  delivered. 

•  In  a  December  1990  report  on  the  Army’s  Air  Defense  Antitank  System,  we 
identified  significant  reliability  problems  that  affected  the  system’s 
operational  availability  (gao/nsiad-9i-5i).  In  our  analysis  of  the  system’s 
1990  operational  tests,  we  found  that  the  system  had  achieved  an 
operational  availability  of  less  than  39  percent  against  a  fielding 
requirement  of  71  percent,  that  the  fire  unit  had  achieved  an  average  time 
between  operational  mission  failures  of  9  hours  against  a  fielding 
requirement  of  60  hours,  and  that  the  weapon  subsystem  had 
demonstrated  8  hours  mean  time  between  operational  mission  failures 
against  A  fielding  requirement  of  92  hours.  Additionally,  there  were 
important  maintenance  and  logistics  issues  that  had  not  yet  been  resolved. 

•  In  April  1990,  we  testified  that  the  Apache  attack  helicopter  was 
experiencing  availability  rates  that  fell  well  short  of  the  system’s  goals  and 
that  the  rates  were  decreasing  as  units  aged  and  accumulated  flight  hours 
(GAOn’-NSiAD-90-33).  Below  the  surface  of  the  low  availability  rates,  we  found 
serious  logistical  support  problems  such  as  undersized  maintenance 
organizations,  weaknesses  in  repair  capabilities,  and  frequent  component 
failures.  While  the  Army  had  initiated  numerous  corrective  actions  to 
improve  aircraft  reliability  and  maintenance  capabilities,  many  of  these 
problems  had  been  discovered  by  the  Army  in  testing  before  the  1982 
production  decision,  but  production  had  continued  without  resolving  the 
problems. 

•  In  reports  issued  in  May  1984,  July  1987,  and  March  1989  on  the 
Peacekeeper  Intercontinental  Ballistic  Missile  program,  we  reported  on 
the  inability  of  the  contractor  to  produce  inertial  measurement  units  at  a 
rate  necessary  to  support  missile  deployment  schedules  (gao/nsiad-84-ii2, 
■oA.  ■S1AD-87-194BR,  and  GAO/NSiAD-89-105).  Not  Only  Were  the  units  being 
delivered  late,  but  some  delivered  units  were  failing  more  often  than 
expected,  and  repair  times  were  longer  than  anticipated.  The  effect  was  to 
reduce  the  number  of  Peacekeeper  missiles  on  alert.  Improvements  were 
implemented,  thereby  reducing  the  inertial  measurement  unit’s  failure  rate 
and  repair  time. 

•  Production  problems  have  contributed  significantly  to  cost  growth  and 
schedule  delays  on  the  B-2  strategic  bomber  program.  To  a  large  extent, 
the  cost  increases  and  delays  attest  to  the  problems  caused  by  numerous 
engineering  changes,  new  manufacturing  technologies,  and  difficulties  the 
contractors  are  encountering  in  manufacturing  low-obsei-vable  aircraft 
that  meet  precise  tolerances  of  stealth  requirements.  In  a  classified  1988 
report,  we  discussed  how  extensive  design  changes  to  the  aircraft  in  the 
early  1980s,  coupled  with  concurrent,  development  and  manufacturing,  had 
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caused  labor  inefficiencies  and  manufacturing  defects.  In  addition,  our 
repoit  discussed  difficulties  in  the  manufacturing  process  and  the 
potential  for  long-tenn  producibility  problems.  For  example,  the  aircraft’s 
leading  edges  were  liandmade  and  required  a  complex  bonding  and  sealing 
process.  In  July  1991,  we  reported  that  the  B-'2  contractors  were  still 
experiencing  significant  production  difficulties  ((iAO/NsiAi)-9i-2ii). 

We  have  reported  several  times  on  the  reliability  of  electronic  warfare 
systems  and  the  test  equipment  used  to  verify  tiiat  the  systems  are 
functioning  properly.-  Our  reviews  across  service  lines  have  disclosed  that 
the  test  equipment  is  often  faulty  and  unreliable.  In  som€^  instances,  we 
found  that  electronic  warfare  systems  considered  to  be  combat  ready 
actually  had  undetected  faults  because  of  unreliable  built-in  test 
equipment.  In  other  instances,  wo  found  that  test  equipment  deployed  with 
the  electroni(  waifaif*  systems  did  not  have  all  the  components  or 
software  necessary  ha  llieir  use  in  diagnostics. 

In  a  198;^  elassilKai  reporl  on  the  Anny’s  Patriot  air  defense  system,  we 
leportx'd  that,  the  diagnostic  s<»ftware  us( d  .vitl;  the  system’s  built'in  test 
i’quipriKmt  ha^  mccessfully  ide  ntified  lauity  ronij)oncnts  only  50  to 
()()  pei'cent  of  the  time.  To  adti/  ess  these  ju'oblf'nis,  the  Army  upgraded  the 
hoftwarc',  addt'd  another  level  of  mainli'nmice,  and  increa>.ed  mainU  nance 
training.  Initial  investment  costs  lor  thinse  changes  was  about  $94  million. 
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